The AI Instant Food Analyzer combines Anthropic's Claude Vision API with nutritional database lookups to transform meal photos into comprehensive nutritional breakdowns. Users upload or photograph their plate, Claude's computer vision identifies each food item and estimates portions, then the application calculates calories, protein, carbohydrates, and fats using structured nutritional data.

The technical challenge was bridging AI vision capabilities (which describe food qualitatively) with quantitative nutritional databases (which require exact food names and portion sizes). The solution uses Claude's structured output capabilities to extract food items in a standardized format, then matches these against nutritional APIs with fuzzy matching and portion estimation algorithms.

• Claude Vision API Integration: Sends uploaded images to Anthropic's Claude API with vision capabilities enabled, using carefully engineered prompts that instruct Claude to identify all visible food items, estimate portion sizes in grams/ounces, and describe preparation methods (grilled, fried, steamed) that affect nutritional values
• Structured Output Parsing: Claude returns food analysis in JSON format containing arrays of detected items with fields for food_name, portion_size, portion_unit, and preparation_method. This structured approach enables reliable programmatic processing rather than parsing free-form text descriptions
• Nutritional Database Lookup: Identified foods are matched against comprehensive nutritional databases (USDA FoodData Central API or similar) using fuzzy string matching to handle variations in food names. Returns detailed macronutrient profiles including calories, protein, carbs, fats, and micronutrients
• Portion Size Estimation: Claude's vision analysis estimates portions based on visual cues like plate size ratios, common serving sizes, and food density. The system converts these estimates to standardized measurements (grams, ounces, cups) that align with nutritional database entries
• Metric/Imperial Toggle: Real-time conversion between kilojoules/grams (metric) and calories/ounces (imperial) using standard conversion factors (1 calorie = 4.184 kJ). Interface remembers user preference via LocalStorage for consistent display across sessions
• Multi-Item Aggregation: Calculates total meal nutrition by summing individual food items, displaying both per-item breakdowns and overall totals. Visual progress bars show macronutrient ratios and percentage of daily recommended values

Image Upload Pipeline: FileReader API converts uploaded images to base64 encoding, which is embedded in the API request to Claude. Supports multiple input methods: file upload widget, drag-and-drop, camera capture on mobile devices, and paste-from-clipboard. Image preprocessing includes compression for large photos (max 5MB) and orientation correction for mobile photos.

Claude Vision Prompt Engineering: The system prompt instructs Claude to act as a nutritional analyst, providing specific output formatting requirements. Example: "Analyze this meal photo and return a JSON array of foods with fields: food_name (string), portion_size (number), portion_unit (string: 'g', 'oz', 'cup'), preparation_method (string). Be specific about portion sizes using visual cues."

API Request Structure: Uses Anthropic's Messages API with vision-enabled models (Claude Sonnet 4). Request includes base64-encoded image, system prompt for nutritional analysis, and parameters for temperature (0.3 for consistent results) and max_tokens. Error handling covers API failures, invalid images, and rate limiting.

Nutritional Data Matching: Claude's food names are normalized (lowercase, remove punctuation) then matched against nutritional database entries using Levenshtein distance algorithm for fuzzy matching. This handles variations like "grilled chicken breast" vs "chicken, breast, grilled" vs "chicken breast (grilled)". Matches below 80% similarity threshold prompt user confirmation.

Calculation Engine: For each identified food item, retrieves nutritional values per 100g from database, scales to estimated portion size, then aggregates across all items. Handles compound foods (sandwiches, salads) by summing component ingredients. Displays results in formatted tables with visual indicators for macronutrient distribution.

AI Vision API Integration Pattern: Demonstrates complete workflow for vision-enabled AI: image encoding, API request construction, response parsing, error handling, and result display. The pattern applies to any computer vision task - document analysis, object detection, image classification - using Claude or similar vision APIs.

Structured Output from AI: Rather than parsing free-form AI responses, the prompt engineering explicitly requests JSON output with defined schema. The code validates this structure and gracefully handles malformed responses. This pattern is crucial for reliable AI integration in production applications where consistency matters.

Fuzzy Matching Algorithm: The Levenshtein distance implementation for matching AI-identified foods to database entries demonstrates string similarity algorithms applicable beyond nutrition - product search, spell checking, duplicate detection, recommendation systems. Shows how to handle real-world data where exact matches are rare.

Multi-Source Image Input: The upload handler accepts images from file inputs, drag-and-drop zones, camera APIs, and clipboard paste events - all routing through single processing pipeline. Pattern ensures consistent behavior regardless of input method while providing maximum user flexibility.

Challenge: AI vision cannot see hidden ingredients or cooking methods
Solution: Prompt engineering instructs Claude to make reasonable assumptions based on visible characteristics (e.g., grill marks indicate grilled preparation) and common preparations (e.g., restaurant pasta typically includes oil/butter). The results page includes disclaimer that estimates don't capture invisible components like oils, sauces, or seasonings, with recommendation to add known additions manually.

Challenge: Portion size estimation from photos lacks scale reference
Solution: Claude uses relative sizing - comparing food items to plates (standard ~10 inch diameter), comparing items to each other, and using knowledge of typical serving sizes. For ambiguous cases, the system displays estimated portions with confidence levels, allowing users to adjust. Added feature: users can photograph food next to common objects (phone, fork) for better scale reference.

Challenge: Nutritional databases use inconsistent food naming conventions
Solution: Built normalization layer that standardizes food names before database lookup. Removes qualifiers (adjectives, brand names), converts plural to singular, maps common variations ("zucchini" → "courgette"), and handles regional differences ("eggplant" vs "aubergine"). Combined with fuzzy matching, this achieves 85%+ accurate matches.

Challenge: API costs for high-resolution image analysis
Solution: Implemented intelligent image compression that reduces file size while preserving food recognition quality. Images automatically resized to max 1024x1024 pixels, compressed to ~200KB using canvas-based JPEG encoding at 85% quality. Testing showed no accuracy loss for food identification while reducing API costs by 70%.

This project demonstrates production-ready AI vision integration. The patterns shown - prompt engineering for structured outputs, image preprocessing for API efficiency, fuzzy matching for data alignment, error handling for AI unpredictability - are essential skills for any developer building AI-powered applications in 2025 and beyond.

The architecture shows how to combine multiple APIs (AI vision + nutritional databases) into cohesive user experience. Each API handles its specialty - Claude analyzes images, nutritional databases provide scientific data - while application code orchestrates the workflow and handles edge cases. This composition pattern scales to any multi-API integration project.

The prompt engineering approach is particularly valuable. By explicitly requesting JSON output with defined schema, the code avoids brittle text parsing and enables reliable programmatic processing. Study the prompts carefully - they demonstrate how to constrain AI outputs while maintaining flexibility for varied inputs.

The fuzzy matching implementation using Levenshtein distance is production code you can adapt immediately. The algorithm handles real-world messiness - typos, variations, regional differences - that exact string matching fails on. Essential technique for any application dealing with user-generated content or AI-generated text.

• Anthropic Claude API (vision-enabled AI models)
• USDA FoodData Central API (nutritional database)
• FileReader API (image upload and encoding)
• Canvas API (image compression and preprocessing)
• Vanilla JavaScript (API orchestration and calculations)
• LocalStorage (user preference persistence)
• Levenshtein distance algorithm (fuzzy string matching)
• WordPress custom plugin architecture
• PHP (API key management and server-side requests)