Evaluation methodology
How FuelForm evaluates nutrition estimates
Published 8 August 2026 · Last updated 22 August 2026
FuelForm ran small internal reliability checks before its App Store release to catch parsing and estimation failures. These checks are useful engineering evidence, but they are not an independent validation of real-world nutrition accuracy.
What was checked
On , two evaluation configurations were run against the same 19-case internal dataset: 10 text descriptions and 9 food photos. The cases were selected by the FuelForm team and were not a random or representative sample.
Each case had a broad, human-set expected calorie band and a minimum expected ingredient count. A case passed only when the response parsed successfully, returned a valid calorie value, met the ingredient-count check, and fell inside the expected calorie band.
Stored results
- OpenAI evaluation configuration
(
food-openai-gpt-4o-mini): 16/19 cases passed — 9/10 text and 7/9 photo. - Gemini evaluation configuration
(
food-gemini-gemini-2.5-flash): 18/19 cases passed — 9/10 text and 9/9 photo.
These are the results stored for those named evaluation configurations. They do not identify which provider or model a later production build uses, and they should not be read as a product-wide success rate.
What a pass means
A pass means the stored output satisfied four internal checks: parse success, a valid calorie value, the minimum ingredient count, and the broad expected calorie range. It does not prove that the calories, portion size, ingredients, macros, or micronutrients were correct.
Important limitations
- The dataset is small, non-random, and assembled internally.
- The expected calorie bands are broad estimates set by people, not laboratory measurements.
- There is no independent ground truth from weighed portions, chemical analysis, or a validated clinical reference method.
- Passing does not establish performance for other foods, lighting, languages, portion sizes, or real-world user inputs.
- Model prompts, providers, and production configuration can change, so these May 2026 checks do not establish current production performance.
How FuelForm uses the evidence
The checks are regression signals: they help reveal whether a configuration can produce structured, plausible outputs on this particular set. In the app, nutrition values remain estimates. The useful safeguard is that a person can inspect and correct the derived meal instead of treating the first output as authoritative.
Questions and corrections
For a factual correction, a source question, or product support, email support@fuelform.pro. Include the page URL and the claim you want us to check.