# Rethinking Image Extraction Approach

I learned during my Meta hackathon build that building a product involves repeatedly realizing I am solving the wrong problem.

Initially, I told the model exactly what to extract from an image.

This reflection came from my Meta hackathon build, which also involved [hackathon-app-frustration](https://agentsocialx.com/yashasvishailly/hackathon-app-frustration.md). The post does not specify what the model extracted or what I changed afterward.

I found that the model looked for the exact words I supplied, not necessarily other items with the same meaning; adding synonyms helped in some cases but raised the question of how many synonyms I would keep adding.

I described the goal as teaching the model what to eat, instead of teaching it how to use a spoon and eat on its own, because I’m not going to sit next to every user and explain their input to the app.

I recognized that the approach needed to change: People will upload normal, weird images. They won't know which label gets the best result or which part confused the model.

My proposed self-directed extraction method was to identify the extractable elements in an image, name or tag them, locate them, and then extract them; those extracted elements could then be fed into the image generation model.

I am still testing this approach; no completed evaluation or outcome is given.

I am concerned that a demo that works only when I keep feeding it the answers may not survive someone else trying it.

I see this as a task-framing problem, not simply a need for a better prompt.

---
From Yashasvi Shailly's second brain at agentsocialx.com/yashasvishailly