“AI will replace data engineers”… “Agents write better code than humans”… “If you’re not vibe coding, you’re already behind”…
Normally, I try to stay as far as possible from statements like the ones above, because I don’t like the hype. Any kind of hype. But, over the past few months, I’ve had variations of the same conversation more times than I can count: at conferences, in LinkedIn DMs, during client engagements: “Nikola, should we let AI write our pipelines?”
And my honest answer is: yes… but not the way you probably think.
Let me put this as straight as possible: I use coding agents every single day. At this point, they write a serious share of my PySpark/SQL code, my pipelines, my validation scripts. I wouldn’t go back, not for a $1.000.000 (ok, maybe for a $1.000.000 dollars:))… But, before you unleash an agent on your production lakehouse, there is one thing you MUST understand about AI-generated code. And, instead of explaining it in theory, let me show you how it almost burned me.
Confidently wrong – WHAT?
There’s one thing about human-written bugs: they are, in a strange way, polite. They throw exceptions, they break the build, and even when they don’t, they usually look suspicious – because when we humans are not sure what we’re doing, our code shows it (a weird variable name here, a leftover TODO there, you know the drill).
AI-generated bugs are a completely different animal. The code arrives in seconds, nicely formatted, with meaningful names and generous comments… and every now and then, it is completely, confidently wrong! No warning, no hesitation, no “I’m not sure about this part”. It simply tells you a wrong answer with a straight face.
I call this phenomenon confidently wrong code. And if you plan to do serious data engineering with AI (in Microsoft Fabric, or anywhere else, to be honest), this is the villain of your story.
Don’t believe me? Let’s look at the example.
Devil is in the… join?!
Long-time readers of my blog may remember my Query folding series, where we hunted various “devils” hiding in the details. Well, let me introduce you to a brand new one.
Imagine a classic medallion setup in Fabric: card transactions landing in a Bronze lakehouse table, and a Silver layer where each transaction gets enriched with merchant attributes: name, commission tier, country. The most boring task in data engineering, right? Exactly the kind of thing you delegate to an agent without thinking twice.
So, I asked the agent to write the enrichment step. Four seconds later, this is what I got:
# Enrich transactions with merchant attributes
df_silver = (
df_transactions
.join(df_merchant, on="merchant_id", how="left")
.select(
"transaction_id", "merchant_id", "amount",
"transaction_ts", "merchant_name",
"commission_tier", "merchant_country"
)
)
Take a look at this code. No, really, take a look, like you would in a regular code review, on a busy Tuesday afternoon, with three more PRs waiting in the queue…
Left join on the business key? Check. Explicit column selection? Check. The notebook runs green, schema is perfect. But watch out, here comes the twist: this code is CORRECT. There is no bug in it! You could stare at this snippet for an hour and find nothing wrong, because there is nothing wrong…in the code.
The bug lives in something the agent had no way of knowing: dim_merchant is a Type 2 slowly changing dimension. It keeps the full history of every merchant – one row per version, with valid_from, valid_to, and is_current columns. Every merchant that ever changed its commission tier (or name, or address) has two, three, five rows in that table…
You already see where this is going, don’t you? Every transaction belonging to such a merchant got joined to EVERY historical version of it. Duplicated transactions. Inflated amounts. My “revenue” grew by 7% overnight, and trust me, nobody in the business questions the numbers when they go UP!
Nothing crashed. No alert fired. The row count went from 8,640 to 9,312, and unless you compare those two numbers, 9,312 looks exactly as plausible as 8,640. This pipeline can run like this for months… until someone from finance asks why the numbers don’t reconcile. Oh, boy.
One missing filter (is_current == True). That’s it. And here’s the part that should really keep you awake: the agent didn’t make a coding mistake at all. It made a knowledge mistake. It wrote a textbook-perfect join over a table whose semantics it simply didn’t know, and it did it with total confidence.
Ok, but my code review process will catch this… Right?
I hear you, I hear you. “We have a rigorous review process, this would never happen to us.”
Let’s be honest here for a moment. Our entire quality apparatus: code reviews, pairing, the famous “LGTM”, evolved to catch human failure modes. Humans write ugly code when they’re unsure. Reviewers, after years of practice, become extremely good at smelling that hesitation.
AI-generated code has no hesitation to smell! It looks exactly as polished when it’s right as when it’s wrong. That gut feeling you’ve built over your entire career (“hmm, this code feels solid”), suddenly carries zero information.
And our little devil from above is even nastier than that. Remember: the code is correct. Unless your reviewer happens to know that dim_merchant carries history (and let’s be real, how well does the average reviewer know the semantics of every dimension table in the platform?), there is literally NOTHING to catch in that diff. The bug isn’t in the code, it’s in the gap between the code and your data. No amount of staring at the pull request will close that gap. (If you want the musical analogy, and this being Data Mozart, of course you do: it’s an orchestra that never plays a wrong note… but occasionally performs an entirely different piece than the one you requested. And takes a bow either way.)
So, what do most teams do? From what I see in the field, they split into two camps. Camp #1 bans the tools and slowly but steadily falls behind everyone else. Camp #2 goes full “vibe coding” mode, ships whatever the agent produces, and accumulates landmines like the one above. Both camps, I sincerely believe, are losing.
What if I told you that there is a third way? And it’s not a tool, it’s a workflow.
Specify. Bound. Validate.
After enough scars, my whole AI-native workflow crystallized into a simple loop that I run for every single task an agent touches. Three simple yet powerful steps.
Specify precisely. The enrichment bug above wasn’t really a code bug – it was rather a specification bug. “Enrich transactions with merchant attributes” left the most important question (which version of the merchant?) unanswered, and the agent didn’t even know there was a question to ask, so it guessed. A proper spec says: join to the current merchant version only (is_current = true), and the row count MUST stay unchanged after the join. Agents are phenomenal at satisfying precise specifications… and equally phenomenal at inventing plausible answers to vague ones. Most “AI wrote bad code” stories are, in reality, “I wrote a bad spec” stories.
Bound aggressively. Don’t let an agent roam freely across your workspace, like a toddler in a toy shop. Give it guardrails: the schemas it must conform to, naming conventions, patterns it’s allowed to use, tables it may NEVER touch. In Fabric terms, this means scaffolding: metadata-driven frameworks where the agent fills in well-defined slots instead of freestyling entire pipelines. A bounded agent can still be wrong, but only in small, findable ways.
Validate ruthlessly. This one alone is the big deal, so please read it twice: never validate AI code by reading it – validate it by interrogating the data it produces! Row-count reconciliation between layers. Uniqueness checks on keys. Distribution comparisons before and after. Our little devil from above survives any code review on this planet… but it doesn’t survive a single assertion: row count before the join must equal row count after the join. One line of code, and the 7% phantom revenue never leaves the notebook. And the beautiful part? The agent will happily write those validation checks for you! You just need to make asking for them a non-negotiable step of the loop.
Specify, bound, validate. Sounds almost too simple, right? Well, the concept is simple. The craft is in the details: what goes into the agent’s context so it stops hallucinating your table names, how to structure task cards so writing specs doesn’t become a full-time job, which validation checks give you the most trust per line of code, and how to wire all of this into Fabric – notebooks, pipelines, Delta tables, the whole orchestra.
Wrapping up (and an invitation)
Now, I’d be doing you a disservice if I pretended a single blog post can teach you that craft. It can’t. That’s exactly why I’m running a live, hands-on workshop: AI-Native Data Engineering with Microsoft Fabric, on September 7, 2026 (5pm CET / 11am EDT / 8am PT), 4 hours, fully hands-on.
This is NOT another “look what ChatGPT can do” webinar. We build a real thing together: a metadata-driven medallion architecture for a card-payments scenario, with coding agents doing the heavy lifting under proper guardrails. You’ll direct agents to scaffold ingestion frameworks, generate Bronze-to-Silver-to-Gold transformations, produce tests and docs, and then you’ll hunt down planted “confidently wrong” code, the same way you’ll hunt it in production. You walk away with a repo of reusable scaffolds, checklists, and validation patterns you can drop into your own tenant on Monday morning.
And, although we build on Fabric, the workflow is deliberately portable, the same patterns transfer to Databricks, Snowflake, or wherever your career takes you next. Platforms come and go… the discipline stays.
The founding cohort is priced at 179 EUR (there’s also a Pro tier with office hours and a 1:1 call, limited to 15 seats, plus a team pack of 5). Full refund within 7 days if it’s not for you, and 12 months of repo updates included.
Two takeaways I’d like you to hold onto:
- AI-generated code doesn’t fail loudly – it fails confidently. Plan for that.
- The engineers who thrive won’t be the ones who ban the tools, nor the ones who blindly trust them, but the ones who learn to specify, bound, and validate.
If you’ve ever merged a green pipeline that turned out to be confidently wrong – or you’d rather never find out how that feels – grab your seat here.
Thanks for reading!
Thanks to Claude for generating these nice-looking visuals.
Last Updated on July 7, 2026 by Nikola
