The Problem
Restaurants in Jordan sacrifice up to 25% of their delivery revenue on discovery platforms like Talabat (a middle-eastern DoorDash equivalent) even when they deliver with their own drivers. Furthermore, they tend to have a weak presence on Google, and tend to rely on sluggish PDF-based services when creating a QR-scannable digital menu. Aklaat (Arabic for "Eats") sets out to solve both these problems simultaneously by creating a unified ordering website that also serves as an in-restaurant QR-scannable menu, from which the restaurant can funnel new delivery orders.
As with any SaaS model, it looks simple at first: I need to scrape many of these restaurants' current internet presence, enrich the data and structure it into a schema using LLM image processing, pass the data on to sandboxed AI agents to create themed matching sites, and then pitch those restaurants with their already-existing data.
The First Hurdle: Data & Scraping
My scraping pipeline follows a series of stages. Each stage represents every scraper that can be parallelized. If a scraper depends on the results of a previous one, it is moved a stage down. So the stages essentially follow a hierarchical pattern with sub-stages, this helps keep things organized (e.g. "1. Facebook Profiles" -> "1.1. Facebook Profile Posts" -> "2. LLM Data Enrichment"). Idempotency is quite crucial as anything could happen: a power outage, an internet cutoff, an unexpected error. Scraping costs money, if your scraping pipeline does not retain a recoverable journal of events, you will face issues.
Summary of important decisions: Using Vertex AI's Batch Mode to save 50% on Gemini LLM usage, outsourcing most scraping itself to Apify with the exception of image scraping, which I used Webshare proxies to do on my own, and using a simple TUI to monitor scraping progress and past runs.
My scraping pipeline has processed 24,000 restaurants and cafes successfully. Of those, a surprisingly low amount had readily available menus to process into a schema: only a couple thousand. I've been cautiously processing a small subset of 200 restaurants into themed websites, currently so I can review and try to sell each site before moving on to new ones, which allows me to build up a cluster of bugs to solve at once, instead of solving bugs only after a huge number of data has been generated. But I initially had two other main concerns: Sandboxing and Theme Correctness.
Website Generation
Sandboxing is the key issue when letting AI agents run freely, after initial research, I decided to use a new and exciting option baked into the linux kernel: Landlock. Thankfully my cloud provider Railway supported it, but I'll run into an issue with this later. For now though, Landlock was good enough at protecting the container from accessing system files, nix configurations, or unrelated project directories. Interestingly, after enough themes, they started to look the same, and after realizing the cause was agents inspecting and mimicking other themes, Landlock already being available made the fix extremely easy to implement.
The Issue With Railway
Limited sandboxing was an issue that was a contributing factor that led to me leaving Railway for a self-hosted VPS, with the two others being Railway's recent ongoing instability and the high costs of running parallel AI agents (upwards of $120 a month even without the agents running 24/7). A VPS gave me a large amount of space, RAM, and vCPU to run heavy AI agents and the necessary chromium plugins they needed to inspect their own work.
"AI Slop" in website design (the "Purple Blue Gradient" phenomenon)
I had four versions of my theming system, the first three built on each other, but the fourth was an experiment that ended up ultimately failing. All three versions of my theming system depend on a primary constraint wherein, a theme contract is provided to the AI and it mainly fills in style and selects which images to use where. This worked great, and I quickly realized that a massive amount of reference photos were available from Google Maps to scrape. After re-initiating my scraping pipeline to collect images of the restaurant and not just menus, the concrete reference data (often 5+ images) very quickly overrode the default infamous Purple Blue gradient that LLM design often falls back to. But the problem was that having a fixed layout made every theme look the same. v4 attempted to solve this by allowing the AI to freely design the layout without a contract, however the AI resorted to unimaginative, bland, and oftentimes broken layouts, which led me to return to v3 and refine it with time instead to have a more flexible layout contract instead.
Internal Tooling (Human-in-the-Loop Review)
I also required an internal tool to review these websites. I initially had sites reviewed manually all together, but I later created an Autoreviewer agent that reviews them before me to filter out immediate and obvious issues, easing the review process. Particular forms of these issues were about attention to detail: not realizing that text must be light even if the background is also light, because the background is in fact transparent, revealing a further background that is actually dark. The Autoreviewer took care of most design issues like that so I could review occasional inconsistencies in the data itself. After each site was reviewed, that would be the last step before it could be sold.
Outbound Sales
I do use sales agents to reach customers; these were adapted from the AI agent framework I used for parallel site generation, but the full details are complex and out-of-scope for this post.
Conclusion
Overall, the project taught me a lot, and refined many of my existing skills, which I hope to take with me into future projects.
I've primarily learned new experience with sandboxing tools, including the linux kernel's Landlock, as well as Prompt Engineering -- as even small shifts in a prompt, or additional reference material, can cause large changes.
Something else that was new to me is avoiding the sunk cost fallacy and being able to revert when a change goes wrong (as I did with v4 theming).
Lastly, I learned the general glue of managing a production SaaS end-to-end that has to reach real people: Payment Gateways (I use Paddle), and Domain Configuration.