Novelty at work is a weird thing

Data practitioners often face genuinely novel problems at work, even when using familiar methods, according to an industry professional reflecting on their experience. The unique intersection of organizational needs and product features means that standard best practices may not apply, requiring creativity and innovation from analysts and researchers. This reality applies to workers at all companies, not just those at major technology firms, as every practitioner occasionally encounters problems that demand novel solutions.

Novelty at work is a weird thing Phew, I spent half a day driving back home from the long weekend with the family, so a short post this week. Over the years I've heard various people tell me and the teams that I work in that we are "on the cutting edge of technology" and "working on problems that have never been seen before". For some questions, like "how do you measure whether users are finding a new feature useful or not?", it's true in that we're measuring some new feature no one's created before. But at the same time, the question is often similar enough to previous questions that I often scratch my head thinking if anything I do is particularly novel. For example, take the new hotness that everyone's working on, all the AI stuff bouncing around. The most recent wave of AI product-ization has only been around for the past couple of years with a ridiculous amount of churn in terms of functionality, interfaces, and design patterns. Since the whole mess is so new, just about every kind of data analysis and research done in the area is very likely some flavor of novel because no one has figured what the best way of doing anything is yet. Everyone is just doing their best, making use of the methods they have available and fumbling around creating new ones to try. As of right now, no one knows what the best way s to measure "how useful do people find these AI tools?". Do you do an in-product intercept survey? Everyone seems to use those thumbs up/down buttons at the end of every response but pretty universally very few people very interact with those on any product. Maybe you have to do observational and diary studies to get the depth we need? Or maybe we send them a standard survey? Or maybe there's behavioral data we can collect that answers the question without bothering the human user? All of the methods are things we've used before. Even the question being asked is largely identical to every other research project around product utility. The only novel bit to the whole affair is just how this particular product is very new and therefore not well understood. You'd find very similar problems trying to understand how users find smartphones useful back when everyone was using either a flip phone or Blackberry. But prior to all the AI nonsense, we'd still deal with "novel" problems in day to day work. New features going out are still generally new features, and even if they are copying an idea from a competitor in some way, there's always in-house creativity that went into the feature to differentiate it from the market. So it can still be genuinely true that no human has ever had to consider the problem that you're very specifically dealing with at work right now. The unique intersection of needs and organizational quirks means that "general best practice" might actually not be the correct way of doing things for your situation. And I think that many data practitioners don't realize the fact that mixed in their everyday work, every once in a while they're going to be asked to do something that is unique and something that no one else has ever done before. Very often it will be some extension of an existing method and problem, but things will be just weird enough that innovation on our parts may actually be useful. I think this point is important for many to understand. It's not just some privileged few who happen to be lucky enough to work at a Big Name company that gets to work on the "new" problems in data work. That idea is patently false. Every one of us, working for our own little organizations, or even on a hobby project, has the occasional chance to do something that demands our creativity as analysts and researchers to overcome problems. These opportunities don't pop up constantly, but if you pay attention you can spot them by how painful it is to crack what should have been a routine analysis. In that struggle to figure out the problem, we actually do something special that no one else has done before. It might not be "write an academic paper" level special, but often special enough that other people who are facing similar problems will appreciate seeing a different perspective. Since we in industry aren't usually thinking about publishing, we don't really pay attention too much to what "the field" is doing. As a result, it's hard to realize exactly when we've done something that expands the field because no one knows where the boundaries are. Very often, we're so busy trying to solve the problems that we're facing, we just simply feel that we're making things up as we go along. I can attest to how in the early days of data science, the definition of "cutting edge methods" was mostly formed from practitioners doing the best they can and then occasionally writing a blog post about what they did or gave a conference talk. Those handful of examples being shared would seed the idea with other practitioners who applied it to their own problems. Repeat that dozens of times and eventually we built up this body of well understood methods. This process is still happening today where everyone is making methods up as they go and only a handful of people bother to write publicly about it. And so, I'll end this long weekend by encouraging everyone to find things they can share about their struggles in doing data work. While it often feels there's nothing new under the sun, I've rarely seen a blog post or talk about such struggles get a poor reception. And who knows, maybe your little post might actually help add to the field's view of how a certain problem can or can't be approached. Standing offer : If you created something and would like me to review or share it w/ the data community — just email me by replying to the newsletter emails. Guest posts : If you’re interested in writing something, a data-related post to either show off work, share an experience, or want help coming up with a topic, please contact me. You don’t need any special credentials or credibility to do so. "Data People Writing Stuff" webring https://github.com/randyau/datawebring : Welcomes anyone with a personal site/blog/newsletter/book/etc that is relevant to the data community. About this newsletter I’m Randy Au, Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. Counting Stuff is a weekly newsletter about the less-than-sexy aspects of data science, UX research and tech. With some excursions into other fun topics. All photos/drawings used are taken/created by Randy unless otherwise credited. randyau.com https://www.randyau.com/?ref=counting-stuff.com — homepage, contact info, etc. Supporting the newsletter All Tuesday posts to Counting Stuff are always free. The newsletter is self hosted. Support from subscribers is what makes everything possible. If you love the content, consider doing any of the following ways to support the newsletter: Consider a paid subscription https://www.counting-stuff.com/ /portal/signup – the self-hosted server/email infra is 100% funded via subscriptions, get access to the subscriber's area in the top nav of the site too Send a one time tip https://www.counting-stuff.com/ /portal/support feel free to change the amount Share posts you like with other people - Join the Approaching Significance Discord https://discord.gg/9xkE2tnhnH?ref=counting-stuff.com — where data folk hang out and can talk a bit about data, and a bit about everything else. Randy moderates the discord. We keep a chill vibe. Get merch https://bit.ly/countingstuff?ref=counting-stuff.com If shirts and stickers are more your style — There’s a survivorship bias shirt