>>4420038>Does it use the context that it is a photo of foodIt does take the other pixels in the image in to account, but answering whether it understands the context is more complicated.
The model doesn't have internal checks like "is this a photo of food" or "what kind of cuisine is this" so instead of thinking in terms of "understanding" it's more accurate to think in terms of combined influences.
If you gave it a cropped image of just the blurry garlic, it would be less likely to produce garlic, because the shape of the item in the center of the image is one influence, but it does take influence from the garlic itself too, so it might still get it.
>pick up on the subtleties>must be able to see>know it's a fork>have an idea>yet it decidedAll the individual pixels (or, sometimes they are grouped up in to blocks) influence all of the other ones at the same time, but not in a strict categorical or traditionally algorithmic way. That's why it doesn't "know" where to put the fork, because it doesn't "know" anything in that rigid way that we're thinking, and the strengths assigned to each possible influence during its training weren't right to turn that input in to the pixels that appear to us as a correctly placed fork. I don't say this to devalue the output, just to explain how it works.
Models are matrices of numbers that perform linear algebra on vectors of input (the input image + sometimes some text), and give vectors of output (the produced image). The maths that it performs is a function, and that function is an estimation of a map on to meaning. The function here is input image + description -> output image, and the model's job is to be the "->".
I'm talking in a somewhat wishy-washy sort of way, because this is a layman discussion, and fair enough that it is so on a photography forum, but if you would like some sign posts to how the nuts and bolts of it work, let me know.
