Apr 8, 2023·edited Apr 8, 2023Liked by Matt Rickard
> For now, data scientists are safe — insights are hard to automatically extract from data — it requires a mix of technical expertise, domain knowledge, and exploration that is tough for models to emulate.
although solving for really valuable insights is often hard. I still see this problem being solved to a very great degree. Drawing out some of the key insights algorithmically is quite simple, most insights are about plotting a distribution and finding significant segment of that distribution w.r.t a north star business metric like OTR. or simply launching the initiative as an experiment and tracking the metrics. I'm sure there are other techniques to draw out insights that require domain knowledge but most product initiatives have a common playbook in terms of what insights they want to measure, funnel for a company remains the same for a long time.
Given enough sample data from previous product initiatives(docs), Generative AI should easily be able to solve "recommend top 5 insight statements given the schema of the data and previous product initiatives", "given an insight statement, generate the sql query for the schema of the table". I could imagine engineers getting most of the insights readily without data scientists given the data they are exporting is sufficiently annotated.
> For now, data scientists are safe — insights are hard to automatically extract from data — it requires a mix of technical expertise, domain knowledge, and exploration that is tough for models to emulate.
although solving for really valuable insights is often hard. I still see this problem being solved to a very great degree. Drawing out some of the key insights algorithmically is quite simple, most insights are about plotting a distribution and finding significant segment of that distribution w.r.t a north star business metric like OTR. or simply launching the initiative as an experiment and tracking the metrics. I'm sure there are other techniques to draw out insights that require domain knowledge but most product initiatives have a common playbook in terms of what insights they want to measure, funnel for a company remains the same for a long time.
Given enough sample data from previous product initiatives(docs), Generative AI should easily be able to solve "recommend top 5 insight statements given the schema of the data and previous product initiatives", "given an insight statement, generate the sql query for the schema of the table". I could imagine engineers getting most of the insights readily without data scientists given the data they are exporting is sufficiently annotated.