Data Science and its relevance, some time down the line
I’ve been working as a Data Scientist for around 2 and a half years now and just wanted to share some thoughts around what I’ve seen evolving in the field along the journey and how I perceive it.
I work in a startup, there’s about 20 of us (Mallzee, check us out!) and as it’s quite common with those sizes, you end up doing a bit of everything. Especially if the company is a data technology company and the products operate in both the B2C and the B2B funnels, the data science folks need to both be the forefront of the innovation in terms of streaming in new ideas and be able to support other teams with the intelligence they need to take business decisions quickly.
On one side, you got the “research” side of things, where you try out new uses for the data you have, you read the new papers out to keep educating yourself, you come up with seemingly crazy ideas and prototype them, you go bug the engineering guys for getting you new data in the database and in the format needed. This is the “new” side of the work, it is the innovative one, the undone. Here, you are an investigator. But also an executor and storyteller: you need to push for the prototypes to become a reality and you need to tell the story on why the new ideas will help the business go further. On this side, you’re pretty much alone, it’s your job. This area is slippery: if you don’t communicate findings well to whom they may concern, if you don’t build stories around them and don’t make them fancy, it’s going to be typically very hard for people to understand what you’re doing and why it helps. Saying this as I’ve learned it the hard way on myself :)
On the other side, you are also a data helper. Lots of work goes into building reliable metrics for the business, and to propel data-driven decision making. Build a culture for looking at the data, and the right one, measuring things right. A mentality for questioning your intuition and designing robust experiments. All this is part old-fashioned statistical work, part business analysis, part finding new ways to communicate a picture from a round perspective, where the point of view is comprehensive: rarely is one single measurement telling you the whole story, and interdependencies may be complicated. That’s where you come in handy. On this side, you cooperate with all (stress: all) other teams. Storytelling here is king again. And you need to understand what all stakeholders need in their job, as well as where the company needs to focus.
As all this demonstrates, and demonstrated me quite extensively now, data science is a bit of an assortment of beasts you need to keep together. A role conceived and brought to life in the shiny realms of innovative Silicon Valley, advertised to the world in noisy fanfare (“sexiest” job, anyone?). But the reality is, it is still in infancy, and quite righteously polluted with misunderstandings. It’s very common to hire someone as a data scientist when what you actually need is just someone who takes care of calculating simple things. Surely enough, what you end up doing in this role can heavily depend on where you work, which sort of industry, size of company, culture. There are data scientists doing hardcore AI all day and those just tasked with reporting work. There are managers who believe these people are there to run queries against a database, and those who make an effort in giving them freedom to explore things and come up with ideas that challenge the status quo. It’s a field which still often suffers from painful misalignments of expectations, I’d quite like to know if the same happened to other roles in the recent past.
However, the misalignment problem does not depend solely on the employers. It’s a pas de deux. Indeed, it is also the responsibility of the data scientist to shape the direction of the field in the company she works for and to push for acceptance. I’ve recently heard someone say in a talk that the most praised skill you need to have as a data scientist is the willingness to fight for your value in the organisation. Obviously it is much easier to affect change in a small environment than in a corporate, highly hierarchical one. On a general note, data science in a big corporation is more standard and paid tools and precise instructions on what tasks to tackle, in a startup is all about open source and trying things out, creating new software with more vague requirements on what to do. In any case I’d say, expecting to do one single thing and to only focus on the cool stuff out there is being naive. The current wave of Machine Learning/AI resurgence is great and I’m convinced it is truly starting to kick in dramatic changes in many things we do as humans, but fine tuning machine learning models isn’t always the best use of time for a data scientist. Sometimes it shouldn’t even be their highest concern. In fact, you got the concept of Machine Learning engineers for a reason.
You role as a data scientist is to produce information out of the data, go hunt for new data and measure the quality of it in a constant cycle of improvement and novelty. The how you produce insight is up to your ability to mix it all together: is a neural network model needed, or a simpler, maybe even heuristic, solution would provide the business better value for cost? This is your job, to decide that, and I’d argue developing a seasoned nose in this respect comes with experience.
I also happen to work in Scotland, which is a quite unique place to be in this respect as it’s a country which is collectively investing in making this field more and more relevant to the economy, mainly through the establishment of innovation centres like the Data Lab, aimed at fostering and facilitating cooperation and pushing for change. To my knowledge, this environment is quite uncommon and blends quite well with the entrepreneurial/startup vibe which is also very lively here. It all makes me sensibly grow, and fast. The advantages of living in a small, well networked, place.
Two or so years ago, data science (in Europe at least) was a pretty unknown bizarre concept. More and more people started then onboarding it, or, should we say, giving it a chance. Sure, as said above the process is still ongoing and nowhere near being finalised, but we’re seeing these concepts getting increasingly applied in the most disparate industries, from healthcare to legal services (!). I’m loving to be each day participating to this, hungry to keep going and looking forward to live what we’ll shape next.