100 Days of Gen AI: Day 20

I only had 40 min available today so I started mapping out a new project. I want to build a comment moderation tool. It’s key features will be:

1. A comment thread interface where you can see past comments and submit a new comment to the thread
2. When you submit a new comment, it is moderated by an LLM before posting
3. The LLM will have a prompt with a list of policy guidelines regarding inappropriate content (I could try to fine-tune a model, but let’s see how far we get with just a prompt instruction)
4. If you submit a comment that is flagged as inappropriate, the interface will tell you which policy it violated, and prevent posting it

I’ll need a datastore and a thin API. I would typically use AWS Lambda + DynamoDB for this kind of project. However since I’ve been using Vercel for several serverless applications so far, let’s see if we can stick with Vercel. It looks like they have a few serverless datastore options: Vercel storage. Vercel KV is probably the right choice for a prototype.

Other thoughts

I’ve been thinking how underlying LLM models will likely become commodities. All the value created will be in how you build on top of them and adapt them to solve enterprise-specific needs or consumer-specific needs. The analogy that comes to mind for me are the public clouds (AWS, Azure, GCP) – a few deep-pocketed companies provide the service like a utility, but all the value is created in how products are built atop them.

I finally read the latest Stratechery article Enterprise Philosophy and The First Wave of AI, and this quote from Satya Nadella resonated with this view:

with scaling laws, as AI becomes more capable and even agentic, the models themselves become more of a commodity, and all the value gets created by you [in] how [you] steer, ground, fine-tune these models with your business data and workflow.

And that is predicated on the data quality used to train these models, and finding the right interface between humans and LLMs.

I also came across this quote which I found funny, but seems true – Jake Heller, the co-founder and CEO of Casetext says all SaaS are “just super-wrappers around databases.” (From Edith Yeung’s weekly newsletter)

Other ideas

Other ideas that came up for me today and I want to keep noodling on:

• How would I design a “Human In The Loop” interface for working with AI agents?
• How will an enterprise fine-tune and expose various models to its employees? (e.g. sensitive HR model, non-sensitive model for all employees, sensitive financials model). How do you control who accesses which models and what data? How do you use data from across your silos / third-party tools to fine-tune each specialized enterprise model? How do you keep them up-to-date with regular fine-tuning?
• If anyone can now use LLMs to generate production-quality code, how will non-developers host this code and run it safely?