The process of IP image protection – an ongoing research project by Esoft
Developing an IP tracking system for images can help us assess the utilization of processed images, as well as being able to identify who ordered the picture, who took it and so on. How do we do that?
In this article we will discuss the current status of our Proof of Concept for, Project Fingerprint/Watermark, wherein we utilize perceptive hashes and neural networks using machine learning to generate unique object-recognition graphs.
To ensure our data is secure and verifiable, we utilize Blockchain to create verifiable ledgers and offer the opportunity to recursively use more Image Fingerprinting based on feature-recognition graphs to further solidify recognition.
Creating imperceptible embedded watermarks leans up against the concept of steganography, the art of hiding secret messages in plain sight within images and other documents.
IP tracking system for images, in 3 steps
Our R&D department started researching relevant literature and results from prior projects to create our own solution for IP image tracking and image recognition.
For that we made a 3 step plan.
- An extendible database for image meta-information
- Image fingerprinting
- Proved ownership of edited images
No technologies were set in stone, the entire idea was to set requirements and then do our due diligence, with research and proof-of-concept projects to find what was the best solution for our company and customers.
Once an image has had items tagged, we can commit them to a normalized graph or fingerprint each object itself.
Step 1: Embed our watermarks
Various image formats act very differently on a code level.
In order to create watermarks that do not deteriorate image quality, and are not easily identifiable or removable, we need to work on tiny portions of the pixels that make up images or hide data somewhere else in the file. Unfortunately, the result is that when hiding data, it is lost when a screenshot is taken or image edits are made.
This meant deciding on a new way to look at the problem so we decided to look into fingerprinting our images instead.
Images that can be viewed on the Internet are easy to re-use and be claimed by others without asking permission. A common method used to stop image stealing is by fingerprinting the image. Fingerprinting can be done in multiple ways. We choose to use hashing algorithms such as SHA-246 and MD5. We needed to use perceptive hashes, so similar or slightly altered images will have similar hashes.
For partial picture detection (cropped images), we created a feature-map of images using machine learning and neural networks. Then alter our networks to create bounding boxes, or sub-images, of each of the objects they recognize, and then create a perceptive hash on these.
Interesting outcome results from this, as we can compare the size of the graphs or subgraphs to determine how an original has been altered – has it been resized, stretched, color changed?
Step 2: Create a database service to store meta-information
Image metadata is formed by a text information details, relevant to the image itself and information about its production. This means we can take each image ID and store them in a blockchain by making a simple database for metainformation.
For partial picture detection (cropped images), we created a feature-map of images using machine learning and neural networks. These networks were originally created by another R&D project for image classification, but we can utilize the object recognition and then alter the networks to create bounding boxes, or sub-images, of each of the objects they recognize, and then create a perceptive hash on these.
If it turns out there are still not enough, we can then alter our networks to create bounding boxes, or sub-images, of each of the objects they recognize, and then create a perceptive hash on these. Then, two rooms with the same setup, but different lamps, for example, will end up having different sub-fingerprints.
The blockchain data is tied to a standardized SQL database, that contains copies of the fingerprint data, along with meta information such as which editor or photographer used the image, which sites it was intended for and any other relevant information.
These pieces of data on our image-use can in turn be used to determine future products of our own.
STEP 3: Proven ownership of edited images
The final step is proving ownership. If we look at economics, people have a ledger to show the transactions. A copy would often be held by a third party for verification.
Proven ownership, uses hashes and trees of iterative hashes to create an independently verifiable ledger for anyone with access.
Esoft is currently working on implementing Blockchain (Blockchain is essentially a verifiable ledger based on hashes) to store the vital bits to prove ownership, e.g. fingerprints, in a blockchain and allow verification after encryption by our partners, giving us severe leverage in case of judicial disputes.
The advantage is, as with the earlier mentioned object recognition networks, that blockchain could also serve in other parts of our supply-chain for tracking and monitoring, so the work for one project also lays the groundwork for future projects.
Everything else is stored in an SQL database, which will also let us know which block in the blockchain its fingerprint data can be found. Copies of the fingerprint data are also in the SQL database, along with meta information such as which editor and photographer used the image, which sites it was intended for, what contents are and any other relevant information for use by our image-editing algorithms that constantly train on existing data to get better.
Images can be squished and altered to form a pattern we can see as a fingerprint for an image.
With proof of ownership, a robust set of fingerprints and an extendible database for image searching, we are now combining it all in one to present you a new solution: The Esoft Image Recognition API.
Image hashes are constructed, and just like a fingerprint can identify a person, an image hash can uniquely identify an image an its creator as well. Fingerprinting our images and save the information in our blockchain will provide us with additional information, which can be found in a regular database.
We can then quickly fingerprinting our images and save the information in our blockchain, then create web-crawlers to crawl real estate aggregators and see our market shares. Or run an image we created years ago (and fingerprinted) through the system to see any information about it.
Would you like to know more about our solutions?
CONTACT US HERE
. . .
Written by Peter Nellemann