How Do Generative Image AIs Work? Searching Query Using Scene Graph Part 2.

Computing Image Similarity with Graph Database

When searching for images similar to a given one, graph databases use similarity measures to find matches

Figure 1. Similarity Calculation with Graph Model

Measure image similarity where a specific image overlaps with a specific SPO (Subject-Predicate-Object)

1. If there is a specific SPO associated with the image, set the similarity high. For example, set the weight to 3 for the SPO with Noun 2 followed by Noun 3

2. 'Img 1' and 'Img 2' have one SPO in common (weight 3): Similarity 3

3. 'Img 1' and 'Img 3' have two SPOs in common: Similarity 2

The more overlaps between the SPOs of the images, the higher the similarity. You can prioritize certain SPOs for higher similarity.

How Do AIs Search for Similar Images Using Scene Graph?

Using the graph model, similar photos are identified based on weighted SPOs. For instance, a photo of a man playing tennis would have a weight of 3 for the SPO ['man' 'playing' 'tennis'].

Figure 2. Original Image

Figure 3. Operation by Similar Measure Types

In this example:

• Similar image 1 is a photo of a man playing tennis, just like the original

• Similar image 2 is a photo of two men in front of a motorcycle

• Despite having more overlapping SPOs, image 2 ranks lower because image 1's key SPO, "man playing tennis," carries more weight, resulting in higher similarity

Advantages of Scene Graphs

Scene graphs offer intuitive visualization, loading results into graph displays for easy searching. While basic SPO search queries are straightforward, complex queries reveal the performance gap between relational and graph databases. Graph databases outperform relational databases with complex queries like “show me pictures of a man wearing Nike shoes playing tennis” due to their efficiency and speed, translating into lower costs.

Graph databases shine even more with videos. Videos introduce a temporal dimension, creating evolving relationships over time, eliminating text ambiguity, and facilitating object categorization. For instance, distinguishing two men in the same video is easier with Scene Graph, which accurately categorizes dynamic objects, reducing storage inefficiencies from redundant instances.

Research and applications for Scene Graphs are still developing. As demand for complex image and video queries grows, managing and using Scene Graphs will become crucial. Graph databases are invaluable for enhancing analytical potential, paving the way for advanced and insightful analyses.

Ready to integrate Scene Graph technology? Contact us today to get started!