Magic's Ultra-Long Context Models: Revolutionizing Software Development with 100M Token Context Windows
公司规模
Large Corporate
地区
- America
国家
- United States
产品
- LTM-2-mini
- Magic-G4
- Magic-G5
技术栈
- Custom CUDA
- NVIDIA H100 Tensor Core GPUs
- NVIDIA GB200 NVL72
实施规模
- Enterprise-wide Deployment
影响指标
- Digital Expertise
- Innovation Output
技术
- 分析与建模 - 机器学习
- 分析与建模 - 预测分析
适用行业
- Software
适用功能
- 产品研发
服务
- 软件设计与工程服务
- 系统集成
关于客户
Magic is a company focused on advancing AI technology, particularly in the domain of software development. They are pioneering the use of ultra-long context models, which can handle up to 100 million tokens of context during inference. This capability allows for more effective code synthesis and reasoning, as the models can consider a vast amount of information, including code, documentation, and libraries, that are not publicly available on the internet. Magic is committed to pushing the boundaries of AI by developing models that can perform complex tasks with minimal human intervention. They are also focused on building supercomputers to support their AI models, partnering with Google Cloud to leverage NVIDIA's advanced GPU technology. With significant funding and a dedicated team, Magic aims to revolutionize the way AI models are trained and deployed, emphasizing the importance of inference-time compute as the next frontier in AI development.
挑战
The challenge in the AI field has been the limited context windows during inference, which restricts the ability of models to learn and reason effectively. Traditional models rely heavily on training due to the short context windows available, which limits their ability to synthesize code and perform complex reasoning tasks. Current evaluation methods for long context models, such as the Needle In A Haystack eval, have inherent flaws that allow models to perform well without truly understanding or storing large amounts of information. These methods often provide semantic hints that make it easier for models to retrieve information, thus not accurately reflecting real-world tasks. Additionally, the memory and computational requirements for handling ultra-long context windows are significant, posing a challenge for scaling and practical application.
解决方案
Magic has developed ultra-long context models, such as the LTM-2-mini, which can handle up to 100 million tokens of context. This allows the models to perform more complex reasoning and code synthesis tasks by considering a vast amount of information during inference. To address the flaws in current evaluation methods, Magic designed HashHop, a new evaluation method that eliminates semantic hints and requires models to store and retrieve maximum information content. This method involves prompting models with hash pairs and asking them to complete a chain of hashes, which tests their ability to attend and jump across multiple points in the context. Magic has also partnered with Google Cloud to build supercomputers, Magic-G4 and Magic-G5, powered by NVIDIA's advanced GPUs, to support the training and deployment of their models. With significant funding and a focus on innovation, Magic is committed to advancing AI technology and setting higher regulatory standards for AI safety and cybersecurity.
运营影响
数量效益
Case Study missing?
Start adding your own!
Register with your work email and create a new case study profile for your business.
相关案例.
Case Study
Infosys achieves a 5–7 percent effort reduction across projects
Infosys, a global leader in consulting, technology, and outsourcing solutions, was facing significant challenges in application development and maintenance due to its distributed teams, changing business priorities and the need to stay in alignment with customer needs. The company used a mix of open source, home-grown and third-party applications to support application development projects. However, challenges resulting from distributed teams using manual processes increased as the company grew. It became more and more important for Infosys to execute its projects efficiently, so they could improve quality, reduce defects and minimize delays.
Case Study
Arctic Wolf Envelops Teamworks with 24x7 Cybersecurity Protection and Comprehensive Visibility
Teamworks, a leading athlete engagement platform, faced rising cyberthreats and needed enhanced visibility into its network, servers, and laptops. With software developers connecting from all over the world, the company sought to improve its security posture and position itself for future growth. The company had a secure platform but recognized the need for a more proactive solution to identify gaps within its technology infrastructure. Data exfiltration and malicious access were top concerns, prompting the need for a comprehensive security upgrade.
Case Study
Sawback IT and Datto Save Client From a Costly Mistake
Ballistic Echo, a software development house, faced a critical challenge when human error led to the deletion of thousands of lines of unique code. This incident occurred before the code was pushed to source control, resulting in significant loss of time, revenue, and work. The previous file-level backup solution they used was slow and inefficient, making it nearly impossible to manually recreate the lost work. The need for a more reliable and efficient business continuity solution became evident to avoid such disasters in the future.
Case Study
Opal Helps Customers Shine Thanks to Datto
SP Flooring & Design Center faced a ransomware attack that encrypted and locked their files. The attack was initiated through a compromised service account set up by an outside vendor. The ransomware infection was isolated quickly, but there was a concern about the extent of the data at risk. The company had backups in place but was unsure of how much information was compromised. The situation required immediate action to prevent further damage and restore the affected data.
Case Study
Zapier Aggregates Multiple Analytics in a Single Dashboard with the New Relic Platform
Zapier, a company that enables non-technical users to push data between hundreds of web applications, was facing a challenge in automating and provisioning servers for optimal performance. The company's environment consisted of 50 Linux servers on the Amazon Elastic Compute Cloud (EC2), a Django application split across several servers, and a backend consisting of a dynamic number of celery task workers fed by messages published to a RabbitMQ cluster. They also maintained a number of internal web services on nginx in front of Gunicorn and Node.js processes. Redis handled simple key and value stores, with logging handled by Graylog2 and ElasticSearch. However, they realized that no level of automation would be sufficient without an effective monitoring solution in place. They needed a tool that could provide immediate alerts when something was breaking and could be easily implemented into their environment.
Case Study
Pipeline Insight Case Study: YARCDATA
YarcData faced challenges in determining the conversion rates of prospects into customers through various marketing efforts and identifying the source of its leads. They wanted to know the percentage of opportunities in the sales pipeline that came from different marketing events, web downloads, or self-sourced sales opportunities. Additionally, they needed the ability to drill down into the data to guide where to allocate more marketing dollars based on the success of previous efforts. Previously, YarcData relied heavily on spreadsheets and Salesforce.com reports, which made it difficult to extract the exact information they needed. This reliance on spreadsheets represented about 70% of their data presentation.
