Define what your parser will accept, measure how often the model breaks it, and separate the prompt change from the defensive parse.
Module · Model API Integration
Lesson 24 of 30 available lessons
Handling a malformed response and designing a contract that produces fewer malformed responses are two different jobs. Connecting the Gemini API in Production covers the first: every branch the API can return needs a handler. This lesson covers the second, and the part that is harder than it sounds — proving the change worked.
⬡ What you'll build
An output contract is the set of shapes your parser will accept. Not "JSON" — the exact fields, their types, and the permitted values.
Writing it down first changes what you ask for. Compare the two requests the Lab actually used. The first names an output format; the second is one:
Analyze whether the following message is a scam. Return a JSON response.
Required output format (return ONLY this structure):
{
"verdict": "LIKELY_SCAM" | "UNLIKELY_SCAM" | "UNCERTAIN",
"probability": <number between 0 and 1>,
"signals": [<array of signal type strings from the taxonomy below>],
"explanation": "<1-2 sentence plain language explanation for non-technical users>"
}
The second is a contract. The first is a hope.
You cannot measure "the output was bad." You can measure a thrown exception.
The Lab's experiment defines failure as one specific, observable thing: SyntaxError events on JSON.parse() in the Cloud Function logs. Not a quality judgement, not a human review — a log line that either exists or does not.
That definition is doing more work than it appears to. It makes the failure rate a count over a period, it makes the measurement repeatable by someone who was not there, and it puts the number in a place that already exists rather than requiring new instrumentation.
Before you change a prompt, answer this: what event, in what log, means the contract was broken? If you cannot answer, you are about to make an unmeasurable change.
The most skippable step is the one that makes everything after it meaningful.
The Lab's baseline — iteration 0, the minimal prompt above — decomposed into three observed behaviours rather than one number:
Splitting the failures by shape is what made the next three iterations targetable. "6% fails" tells you to try harder. "4% is wrapped in fences and 2% has prose around it" tells you exactly what to forbid.
A sample of what iteration 0 produced — the model reasoned correctly and decorated the answer, which JSON.parse() rejects at position 0:
```json
{"verdict":"LIKELY_SCAM","probability":0.85,"signals":["urgency","financial_request"], ...}
```
The experiment ran four prompt structures, not three. Reading them as a sequence is the point:
| # | What changed | Parse failure rate |
|---|---|---|
| 0 | Minimal — "Return a JSON response" | ~6% |
| 1 | Schema described in prose (a field list) | ~4% |
| 2 | Exact JSON shape embedded in the prompt | ~2% |
| 3 | Schema + explicit format suppression | ~0.8% |
| 3 + | Iteration 3 + cleanGeminiOutput() pre-parse layer | ~0% |
0 → 1 is the instructive one. Describing the schema in English moved the rate from ~6% to ~4%. The lab's own verdict: "Marginal improvement — the prose description of the schema did not eliminate decorated output." The obvious fix bought the least.
1 → 2 halved it. Giving the model a literal template to mirror, rather than a description to interpret, is a different kind of instruction.
2 → 3 added the explicit prohibition: "Do not wrap it in code fences. Do not add any text before or after the JSON." The lab records this as "not redundant."
3 → 3+ is not a prompt change at all, and that distinction is the next section.
The last row of that table comes from code, not from the model:
function cleanGeminiOutput(raw) {
let text = raw.trim()
// Strip markdown code fences
text = text.replace(/^```(?:json)?\s*/i, '').replace(/\s*```$/, '').trim()
// Extract first JSON object
const firstBrace = text.indexOf('{')
const lastBrace = text.lastIndexOf('}')
if (firstBrace !== -1 && lastBrace > firstBrace) {
text = text.slice(firstBrace, lastBrace + 1)
}
return text
}
Keeping these two layers separate in your head — and in your measurements — matters, because they fail differently. A prompt constraint reduces how often the model misbehaves. A cleaning layer absorbs the misbehaviour that remains. The first can regress silently when the model is updated; the second cannot, because it is your code.
The lab's conclusion is the pattern worth taking: "tight prompt constraints (reduce failure frequency) + pre-parse cleaning (handle residual failures). Neither layer alone is sufficient for production reliability."
If you collapse them — clean the output and never measure the prompt — you lose the ability to notice that the model got worse. Your cleaner will quietly absorb a rising rate until it meets something it cannot fix.
This is the section most write-ups omit, and the one that separates a measurement from a marketing claim. Every limitation below is a property of the source, read from it directly.
~ and should be read as an approximation, not a computed statistic.~0% is a count, not a rate. It records zero parse failures across logged production inputs over an unstated denominator. Zero observed failures is not the same as a demonstrated zero failure rate.None of this makes the experiment worthless. It makes it honest evidence of a direction rather than a universal constant. The number you should trust is the one you measure on your own traffic.
One more thing worth knowing if you read the corpus: the failure report Gemini API Returns Malformed JSON describes the same ~6% baseline but credits the recovery to the cleaning layer, where the lab credits most of it to prompt iteration and the remainder to cleaning. The lab is the finer-grained account. When two of your own documents frame the same number differently, say so rather than picking one silently.
You changed the prompt. The rate fell. That is two facts and one assumption.
The assumption is that nothing else moved. Over a multi-month window, plenty can: the input mix changes, the model is updated underneath you, traffic shifts to different users. A before/after comparison across different periods is suggestive, not controlled.
What you can honestly claim: "After this prompt change, the observed parse-failure rate over this period was X, down from Y over the prior period." What you cannot claim without more work: "This prompt change caused a Z% reduction."
Change one thing at a time where the system lets you. Where it does not — as with iteration 3 — say which things changed together.
Reading this lesson does not demonstrate the capability. Producing the artefact does. The capability asks for a before/after parse-failure rate with the prompt change that caused it — a measurement and a diff.
Step 7 is the one that makes the artefact worth reading. A before/after with no stated denominator is the same shape of claim this lesson has just spent a section qualifying — and you now know how that reads to somebody else.