Scale AI

Yuka, a mobile application that provides health impact information for food products and cosmetics, faced a significant challenge in managing its rapidly growing database. The database, which already contained over 4 million products, was expanding at a rate of approximately 1,200 new products daily. Yuka's small team was unable to manually review each new product added to the platform, a process that often required multiple transcription tasks. The application initially used OCR to scan product images for nutritional information and ingredients, but this process was not always accurate. OCR struggled with images featuring inconsistent lighting, obstructions, or irregular text surfaces. As a result, about 60% of the images submitted to Yuka needed to be outsourced to a human annotator. This was a daunting task for Yuka's small team, especially considering their goal to provide a product's health score within 2-3 hours of its addition to the database.

OpenSea, the world's leading marketplace for non-fungible tokens (NFTs), was facing a significant challenge in detecting and mitigating copymints and fraud. Copymints are duplicates or imitations of popular NFTs, which can deceive users, especially those new to the world of NFTs. Trust and safety are crucial for welcoming new people into the Web3 ecosystem, and OpenSea was looking for a vendor to help advance their detection and removal capabilities. The team had already used rule-based systems to capture forms of deception, but it was a challenge to achieve the desired speed, recall, and precision needed to effectively address fraud in the marketplace.

Advanced.farm, a company focused on automating agricultural tasks using robotics, was facing a challenge in refining its apple-picking capabilities. With numerous apple varieties and a short picking season, it was difficult to maintain pace. As they developed their computer vision machine learning (CVML) capabilities for apples, they needed a labeling solution that would allow them to regularly create new projects and receive a quick turnaround on labeled images. To succeed in their first apple-picking season, it was crucial for them to quickly process a large number of images through the annotation pipeline, adapt to the changing variety of apples, and ensure that their models were as accurate and efficient as possible on real data.

Flexport, a technology platform for global logistics, was facing a significant challenge in processing logistics documents such as bills of lading, commercial invoices, and arrival notices. These documents, which are critical for clearing shipments past customs and establishing ownership of goods, were traditionally processed using template-based and error-prone OCR (optical character recognition) solutions or manual labor. This method was not only time-consuming but also prone to errors, leading to delays in cargo movements and slowing down internal operations. Flexport realized the need for a machine learning-based document processing solution that could automate the process and extract valuable information accurately in seconds. However, the challenge was to find a partner with deep expertise in AI and machine learning who could operationalize this solution without Flexport having to build out a team of machine learning engineers or data scientists.

Nuro, a robotics company specializing in autonomous vehicles for delivery services, faced a significant challenge in identifying infrequent but meaningful scenarios in their training data. The company's autonomous vehicles, designed to deliver goods from produce to prescriptions, needed to be able to identify and respond to a variety of obstacles, including pedestrians in unusual postures, animals, occluded and backlit pedestrians, and infrequently encountered vehicles such as excavators. However, these labels were not present in the ground truth of their training data. The company's internal tool was only able to identify a limited number of these scenarios, falling short of the thousands of images that needed to be identified and labeled for comprehensive training of their autonomous vehicles.

Orchard Robotics, a company providing AI-first precision crop management solutions to farmers, faced a significant challenge in collecting and utilizing precision data across vast commercial orchards. The company developed tractor-mounted, AI-powered camera systems to collect precision data about every tree. However, the company needed to accurately count every fruit on every tree, a task that proved to be incredibly difficult and tedious, especially when the fruit was small. As a small team, Orchard Robotics struggled to scale these annotations in-house. They initially tried using three other major data-labeling services, but they could not achieve the consistent quality they needed. The quality varied dramatically between batches, and they could not provide feedback to the annotators on the quality of the labels. These platforms also did not offer ellipses as an annotation type, forcing Orchard Robotics to rely on bounding boxes, a less-than-ideal option when labeling spherical fruit.

Ambi Robotics provides AI-powered robotic systems to customers, enabling them to scale their operations and handle increasing supply chain demand. The company's machine learning (ML) system is responsible for identifying an object and its location, and moving the robot hand to that location to grasp the object. The pick success rate, which is how often a robot successfully picks up an object, is the most important marker of success. However, Ambi Robotics faced a challenge in obtaining high-quality annotations for their data, which is crucial for improving their models. Initially, the company was managing the annotation process in-house, but this approach was not scalable for the amount of data they needed. When working with new clients and locations, Ambi Robotics would sometimes see lower pick-and-place success rates, simply because the environment looked different. The best way to improve performance was to mine data from the new location, annotate it, and then retrain their ML model. However, the company lacked the infrastructure to process this large quantity of data on a recurring basis.

Kodiak Robotics, an autonomous technology company, is developing self-driving capabilities for the long-haul trucking industry. The company uses a unique sensor fusion system and a lightweight mapping solution to navigate highway driving and deliver freight efficiently. However, the company faced a significant challenge in training its software to handle rare scenarios, such as pedestrians walking on the highway. These edge cases are crucial for a production-level autonomous vehicle system, but collecting enough real-world examples to train the models reliably was proving difficult.

CellarEye, Inc. is a company that aims to revolutionize the management of private and professional wine collections by leveraging state-of-the-art Computer Vision (CV) and Artificial Intelligence (AI) technologies. Their goal is to provide a seamless management system that automatically tracks each wine bottle in a cellar, storing both the brand and location into inventory tools without manual entries. However, the team at CellarEye faced a significant challenge in realizing their vision. They needed to develop a reliable object detection model to recognize and track wine bottles as they were registered to and removed from the inventory. The cellar environment, with its thousands of wine bottles, presented a complex scenario with numerous edge cases. The company initially struggled with bad or inconsistent annotations, which made achieving an accuracy rate of over 80% a challenge. They needed a better way to detect problems with their data, understand their model failures, and enable their Machine Learning (ML) team to collaborate with their annotation team to catch labeling mistakes faster.

Goodcall, a company providing businesses with intelligent phone agents, faced a significant challenge in managing and annotating the high volume of data generated by their chatbots. The chatbots, which use automatic speech recognition (ASR) to convert speech-to-text and AI analysis to interpret customer requests, required regular fine-tuning with real-world production data. However, the process of labeling this massive amount of data with high-quality annotations was time-consuming and resource-intensive. Furthermore, Goodcall was unable to match the scale of available data due to their in-house data annotation process. This meant that every piece of unlabeled data was a missed opportunity to improve their models. To enhance model performance and customer experience, Goodcall needed a scalable, sustainable approach for labeling large quantities of data.

Kaleido AI, a Vienna-based company, is dedicated to simplifying complex technology by creating tools that accelerate workflows and foster creativity. The company introduced remove.bg, an automatic image background remover, and Unscreen, a video background remover, which gained immense popularity and led to its acquisition by Canva in 2021. However, Kaleido AI faced a significant challenge in improving its machine learning models. The company's models required a large volume of high-quality data, but they encountered several edge cases in a specific segmentation task where their model performed poorly. Collecting and labeling tens of thousands of real-world images with a large diversity of patterns, images, backgrounds, and textures was difficult. Open datasets did not have enough high-quality images of this particular class. Kaleido AI initially relied on real-world data to train its segmentation models, but this approach was complex, resource-intensive, and costly.

SAP, a leading software corporation, was facing a challenge in improving its products around document processing, particularly those dealing with invoices, purchase orders, and payment advices. The team had a vast collection of customer documents but required a partner to create a comprehensive dataset to enhance their accounts payable products while respecting data ownership, privacy, and sensitivity. The need for high-quality data was paramount for performant models. SAP needed superior quality training data to train models for processing and extracting crucial information from purchase orders and invoices in English, German, and Spanish. The variability in customer data, with some providing thousands of documents a week and others taking months for a fraction of the same volume, added to the complexity of the challenge.

Velodyne Lidar, a company that builds lidar sensors for safe navigation and autonomy across various industries, was facing a challenge in managing and selecting relevant training data from the large quantities of sensor data they collected. The data team found it relatively easy to classify common indoor robotics scenes as these scenarios made up a large portion of the datasets captured on their test robots. However, finding rarer scenarios, such as a warehouse employee stacking boxes on the top of a scissor lift, proved to be a difficult task. The team was in need of an out-of-the-box solution that could provide the necessary tools for efficient data selection and management.

Voxel, a company leveraging AI and computer vision to manage risk and operations, faced two significant challenges. Firstly, they needed to maintain high-quality training data for their computer vision system. Secondly, they sought to automate their labeling process for faster throughput while retaining their in-house annotation team. Voxel had already invested in an in-house annotation team of subject matter experts, but they were struggling with efficiency in their labeling operations. They had been using an open-source solution, Computer Vision Annotation Tool (CVAT), which was causing bottlenecks as they increased the volume of annotations needed for model training. From an operational perspective, Voxel found it difficult to efficiently collect data and insights on the data labeling process, leading to significant manual effort. The tool couldn’t effectively link data quality to individual annotators, making it hard to identify the cause of low-quality labels. On the engineering side, Voxel had to custom-build data pipelines for new customer projects, a process that took multiple engineers four weeks for each project.

Brex Inc., a financial service and technology company, was facing a significant challenge in automating its financial workflows. The company's goal was to provide an all-in-one finance solution for businesses, including features like Bill Pay, which allows businesses to manage and pay their bills in one place. However, much of the industry still relied on manual, error-prone workflows, particularly in document processing. Brex found that traditional OCR solutions were not reliable enough. The processed information from uploaded receipts or bills was often incorrect, requiring verification and re-typing. Even solutions that claimed to use machine learning were not achieving high enough accuracy and required substantial upfront work from the Brex team to set up templates. The challenge was to find a solution that offered high accuracy and low latency.

The Datta Lab at Harvard Medical School is engaged in studying the neural mechanisms associated with behavior in rodents. Their research involves recording the behavior of mice using cameras and measuring their neural activity using neural implants. The challenge lies in the analysis of this data, particularly in interpreting the behavioral data. This requires the researchers to label the poses of the mouse over time. While machine learning models can automate this process, a significant amount of video footage needs to be manually annotated first. This annotation process is time-consuming and detracts from the time that researchers could be spending on other aspects of research that require their expertise. The lab was in need of a solution to speed up their data annotation process.

The TimberEye team faced a significant challenge in enhancing their mobile application's log scaling capabilities. The app, which uses computer vision and LiDAR mapping technology, was designed to help lumber suppliers and buyers categorize and scale logs faster, more safely, and with better accuracy. However, the team wanted to experiment with an instance segmentation model to further improve the app's scaling capabilities. The process of annotating images for segmentation proved to be a daunting task. TimberEye CEO and Founder Scott Gregg attempted to annotate a segmentation dataset on his own, but after three days and only 1,000 images labeled, he was burned out. The process was significantly more challenging and time-consuming than annotating images for object detection, requiring 100-200 mouse clicks per image instead of just 4. The team was overwhelmed and stuck, with only 5% of the dataset they needed to annotate complete.

Vistapath, a pathology lab, was facing a significant challenge in the grossing process, a critical step in diagnosing diseases like cancer. Grossing involves assessing and documenting the physical characteristics of tissue samples, a process that is prone to human error and can lead to misdiagnoses. Vistapath aimed to reduce these errors by leveraging computer vision and artificial intelligence. However, they faced a problem in developing a robust tissue detection model. The model required hundreds to thousands of accurately annotated images, a task that required a tool that could be easily used by their histologists and experts. Initially, Vistapath used an open-source annotation tool, but it lacked automation and scalability. They then tried a tool with more automation, but it failed to meet their security and compliance requirements. Therefore, Vistapath needed a partner who could provide an annotation automation tool that could meet their strict security and compliance requirements.