Not only are the Electro-Optical sensors of aircrafts and satellites improving, but image sensors and microscopes are also seeing advancements. Microscopes are becoming increasingly prevalent in various fields, with electron microscopes gaining popularity and becoming more affordable. Additionally, atomic force microscopes are becoming more widely available in the market. Deep Block will delve into the challenges faced when integrating machine learning technology to automate microscopic analysis, and how Deep Block successfully overcame these obstacles.
Analyzing micrographs using machine learning techniques presents several challenges due to the unique characteristics of microscopic images. Here are some of the difficulties associated with applying machine learning to analyze micrographs.
- Data Size
In some cases, micrographs can be 10GB or more. When using image export software provided by microscope developers, the file export formats are limited, and the JPEG compressed images they provide are still very large. Creating machine learning software to process these high-resolution images is really difficult than expected. This is explained in detail in our blog post. - Data Dimension
Images taken by a scanning electron microscope are taken in a 3D format. This image is analyzed in the form of a 3D image or converted to a 2D image and analyzed.
In contrast, pictures taken with a transmission electron microscope are taken in 2D.
Because most micrographs are high resolution, they are saved in a format called tiled TIFF. This format is used for saving extremely high-resolution images, and these tiled TIFF images are different from regular TIFF images, so they cannot be opened in normal image viewers. - Image Resolution
The purpose of using a microscope is to find small objects. Therefore, microscopes with good resolution are being used more and more, and the resolution of pictures taken with these microscopes is also increasing. The resolution of a single microscopic photo can reach 100K, and we have already explained that it is very difficult to detect something from HIGH resolution images. - Frontend
Developing an image viewer capable of handling ultra-high resolution images is a formidable challenge. To create a web-based application, seamless communication between the server and the front-end is essential. Additionally, as tiled TIFF files are not supported by browsers, the micrograph file format must be converted on the server. It's important to note that rendering of images larger than 120MB is not possible in web browsers. This prompts the need to display very high-resolution images in smaller sections. Consequently, the front-end must continuously load and process these smaller image pieces. Failure to properly handle these fragments could potentially overload the browser's memory limit. - Speed of Inference
In order to analyze microscope images using computer vision models, images must first be divided into smaller pieces. This requires a significant amount of file system access and increase the overall data volume. However, many natural scientists are not well-versed in effectively processing large amounts of data, making it challenging to develop software that can quickly analyze high-resolution micrographs. In order to process such data, the help of experts with a good understanding of software engineering and computer science is needed, and Deep Block's team is comprised of such experts. - Computation
The size of a single microscope image can sometimes exceed 10GB. If these large images are not stored carefully, it won't be long before storage capacity becomes full. In addition to this, the creation of high-resolution image analytics, such as Deep Block, necessitates the use of an inference server capable of running deep learning models, storage server, and a front-end application that can render these high-resolution images flawlessly. To accomplish all of this, it is imperative to employ parallel processing technology, various cluster computing technologies, and the latest web technologies. This is not a task that can be undertaken by a single individual; it requires the expertise of computer science professionals. - Domain Expertise of Dealing with High-resolution photograph
Processing ultra-high resolution images requires the professional team that has extensive experience working with such data. It is not a task that can be accomplished easily by simply applying a machine learning model. Analyzing high-resolution images and incorporating machine learning technology into the process present distinct challenges that necessitate the skills of an expert who can effectively handle both aspects. Deep Block has been successfully analyzing this type of image using machine learning for the past six years, making them the ideal choice for your needs.