Hello everyone, there’s a reason why I’ve been inactive for a year.
I’ve come out of Monk mode (long, uninterrupted work blocks) and I’ll explain what I’ve been doing.
At the end of 2024, I created a Custom GPT AI Agent that would:
Find the top 20 most in-demand non-technology remote roles
Find the top 20 most in-demand technology non-coding remote roles
Search the internet and retrieve the in-demand roles you selected from Specialty Recruitment Agencies.
Why only search among these agencies?
Over years of applying to roles, I have found working with those agencies to have consistently much higher rates of success getting roles, providing prompt recruiter communication (They don’t ghost you), and feedback from Specialty Recruiters than from Job Boards or internal company recruiters.
Everything was going great (or so I thought). The AI Agent was generating job URLs, job descriptions, recruiter contact names, phone, numbers, and email addresses. Perfect!
Then I went to validate and test the job data…everything was being hallucinated. What an embarrassment! No bueno!
I quickly went back to square one.
Since the AI Agent was relying on the LLM for the data, I had to find a way to scrape the data from the specialty recruitment agency websites.
That was the hard part! Every agency has different sitemap schemas for their job URLs, and some use other methods to show the job listings. So, by trial and error, I came up with methods to gather the job URLs from over 150 agencies worldwide with over 300,000 active job listings and counting!
Then I created three separate apps and placed them in a data pipeline workflow called:
App1 - URL Discovery (Sitemap Scanner)
What it does: Finds job listing URLs without visiting the actual pages yet.
App2 - Page Scraper (Job Data Extraction)
What it does: Actually visits each URL and extracts the real job information
App3 - Post-Processing (Data Cleanup & Enrichment)
What it does: Cleans up the scraped data and adds extra intelligence.
This workflow is run daily and scrapes the agency sitemaps for new job listing URLs.
The scraped and cleansed job URLs with job descriptions, locations/ remote, salary, employment type (contract or perm), and recruiter contact info goes into a RAG database where the AI Agents:
can extract the information for the app User through a chatbot and they are EXPLICITLY prevented from hallucinating data.
The Chat has menu selections, it can also answer questions, or you can query the database further by asking the agent to provide specific job URL information.
Here’s some of the important insights I’ve learned which have been corroborated by technology companies heavily involved in AI workflow adoption:
“The way the algorithms work, the way they can make this massive amount of AI and data crunching working is their job number one is to goal seek and get you an answer.
And that's why when you just plop into ChatGPT, sometimes it hallucinates when it doesn't know the answer because it's not just that it's hallucinating, it's goal seeking. It's getting you the best answer it can. And if it doesn't know the answer, it makes it up.”
- Founder and CEO of SaaStr, Jason Lemkin
“So what are the folks that are actually deploying automation at scale doing?
They're finding ways to mix and match deterministic workflows with agentic workflows and do those together. And so they know how to break down the steps in a workflow bit by bit.
And they recognize, here is a place where I want it to do the same thing every single time… especially when a deterministic workflow is perfectly good at doing that use case. It is good, it's fast, it's cheap. And so those are the places where you want that to work 100% the right way.
However, there are certain use cases that deterministic workflows simply can't do… And so the people that are getting the most out of AI automation today, they know how to mix and match determinism with agentic workflows to really get a lot of power out of these tools.”
- Zapier CEO & Co-Founder, Wade Foster
• The best practitioners decompose a process into sub-steps.
• Use deterministic automation where correctness is required.
• Insert agentic automation where you need something creative/synthetic.
• That “mix and match” approach is, in his view, what’s delivering real value from AI automation today.
“Building multiple integrated products in parallel leads to stronger product-market fit than building a single feature/product sequentially.
Customers experience workflows, not products.
So, if you ship one isolated tool, users evaluate it as:
‘Does this replace my current workflow?’
Almost always → No
Because workflows are bundles.
Breakthrough companies don’t build one product. They build a stack of connected products simultaneously. Not randomly — but around a single job-to-be-done.
Parallel product strategy removes the integration tax.” - CEO of Rippling, Parker Conrad
Our “Secret Sauce” is the over 300,000 live recruiter-owned job URLs updated daily and sourced from over 150 Specialty Recruitment Agencies Worldwide. Which are both proprietary and live data.
“You can put a relatively commodity model in front of it and get much better results than the most cutting-edge model that does not have access to the proprietary or live data.”
- a16z Partner, Anish Acharya
Here’s a Use Case that employs this strategy:
“A&O Shearman has created an artificial intelligence tool to speed up work performed by more senior lawyers, as the “magic circle” firm attempts to generate revenues from the disruption of the legal industry.
The tool, created in collaboration with AI start-up Harvey, is focused on time-intensive, low-billing tasks including in areas such as antitrust and fund formation that require senior associate or partner input and oversight.
The antitrust element of the model, which A&O Shearman will use itself and sell to other firms for a fee, uses a company’s financial information to assess which of the more than 130 jurisdictions a client might need to make a regulatory filing in for a merger. It then identifies what data is missing and drafts the information requests for each party.
Such exercises often take hours of associate time and input from more senior lawyers because of the high stakes involved when considering a transaction.”
We will be now opening Job-Genie.ai for the first round of Alpha Testers that will incorporate improvements to the app’s functionality. We will follow with a second round of Beta Testers, before releasing the app to the market.
Both Alpha and Beta Testers will be able to get their data and results for FREE!
Here’s the Job Genie Road Map before releasing the paid version to the market:
Job-genie.ai achieves workflow replacement when it can:
Continuously surface high-fit agency roles from our proprietary and live data
Let users shortlist and qualify roles quickly
Generate and save job-URL-specific resume versions
Provide a guided apply and exportable package
Track applications and outcomes, and use those outcomes to improve the next cycle
If this looks interesting to you and want to become part of a revolutionary software product, join the onboarding list: