Etched

A Big Picture Problem 🗺️

The Quandary of Progress

The Dilemma

• Current State: The exponential growth of AI is causing data centers to consume far more energy and has created an unprecedented demand for specialized computing power. As of 2022, the International Energy Agency estimates the energy demands and subsequent emissions output from data centers and data transmissions networks were 1% - 1.5% of global energy and emissions. However, a separate analysis from The Guardian estimates data center emissions may be more than 6x this reported amount, which underscores the industry’s needs for more efficiency, better emissions tracking and abundant clean energy.

• Complications: The field of AI is currently resolving bottlenecks and shortages in both energy and chip manufacturing. Since these bottlenecks and shortages are systemic, it will take years to fully alleviate them. In the meantime, research teams are improving the efficiencies of their existing software systems, but society’s computing paradox is that more efficient compute leads to more demand and consumption, which then drives more energy and ultimately more emissions over time.

• Consequences: The pressure to balance increasing compute demands with sustainability efforts will only intensify as the years go by. While software optimizations can provide short-term relief, they must be matched by breakthroughs in hardware efficiency to prevent runaway energy usage and emissions growth. As data centers expand and AI workloads grow, breakthrough chip designs and system architectures will be essential to ensuring the industry can scale sustainably.

Searching For A Solution 📡

Etched

The Basics

• Mission: To build the hardware that powers artificial superintelligence.

• Summary: Etched is developing a chip called Sohu that is tailor-made to improve the performance and efficiency of running AI models on the latest hardware architecture.

• Year Founded: 2022.

The Framework

• Solution(s): Etched is reimagining an optimal chip architecture to perform AI inference tasks on transformer models with their Sohu chip. General-purpose GPU (Graphics Processing Unit) hardware can only dedicate a fraction of their transistors to performing computational inference because the chip also needs to handle other tasks like graphics rendering (as the name implies). However, graphics rendering and other sorts of tasks are uncommon on transformer models, and since Sohu is building their chip specifically for transformers, they can theoretically remove several components in a general-purpose GPU chip and replace them with components for more computational inference in the same silicon area. The result is a specialized chip to run AI workloads very effectively, but is not setup to do other tasks.

• Strategy: In addition to their specialized transformer chips, Etched plans to provide server hardware and a software stack to customers interested in their product. They are offering these additional products and services so organizations can seamlessly switch some or all of their existing infrastructure to the Etched system with minimal disruptions to their existing operations.

• Signal: Etched mentioned in recent press release that Sohu demonstrated at an order of magnitude improvement over current chips in metrics such as throughput (tokens per second), latency (time-to-first-token), and cost (per token) for popular models like Llama 70B. Etched did not provide enough context for us to thoroughly vet these performance claims in their press release, and we did not find an independent source replicate similar performance with their chips. The organization did mention they are batching data together to process more inputs at the same time, which is known to increase a system’s throughout. However, it is also well known that a tradeoff to this decision can be longer latency in a system. Etched has presumably counterbalanced these tradeoffs with its architectural design, but the true performance and value of their product is yet to be determined.

The Team

• Founder(s):

  • Chief Executive Officer - Gavin Uberti.

    • Previously:

      • Software Engineer @ OctoML.

  • Chief Science Officer - Chris Zhu.

    • Previously:

      • Math and High Performance Computing Researcher @ Harvard University.

• Headquarters: San Francisco, California, United States

• # Of Employees: 11 - 50

Click the button below to request an introduction to the founder

The Channels

• Socials: LinkedIn.

• Website: https://www.etched.com

• Resources: YouTube.

Risk Alerts ⚠️

The Threats

• Commercial Risk: Etched is betting its livelihood on the assumption that transformers will remain as the backbone of AI models for the foreseeable future. If engineers discover transformer architectures have a fundamental limitation to future scale and performance over the next few years, then the industry will be replace transformers with something else, and Etched's specialized hardware may become obsolete. Even in the event where transformers remain as the industry’s hardware of choice for years to come, Nvidia and other market incumbents have the talent, capital and motivation to replicate, acquire and surpass whatever products Etched produces that gain traction amongst its customers.

• Technical Risk: Since Etched is building a chip for a specific type of architecture, they need to spend an inordinate amount of time on all sorts of tests to ensure their hardware design and manufacturing process works seamlessly with the latest transformer updates, even if the changes are significant departures from previous versions. This means they cannot fully dictate their own product roadmap, which can lead to significant ramifications over time. Furthermore, any unforeseen issues the team encounters in their fabrication process or in discrepancies between their simulated and real-world chip performance can erode the trust and patience of their prospective customers.

• Comparable To:

  • Groq.

  • Cerebras.

  • Lightmatter.

Deal Tracker 🧮

The Cap Table

• Series: A.

• Raised: $125 million USD.

• Investors: Led by Positive Sum and Primary Venture Partners. Participation from Lightscape Partners, Hummingbird Ventures, Oceans and others.

Market Insights 💡

The Landscape

• Recent News: According to CBInsights, nearly a third (31%) of all venture funding funneled into AI startups in Q3 2023. CBInsights considers both software and hardware applications under their AI category, though a closer approximation to momentum in the hardware sector of the market was reported during Nvidia earnings, when they mentioned their data center business unit grew its revenue over 400% from the year prior, to $22.6 billion USD. This momentum is driven by every corner of the economy: the venture capital ecosystem that wants to participate in the next technological wave; companies and universities that want to experiment with, if not outright transform their operations for profitability, relevancy and other reasons; countries aiming to secure economic and national security advantages for the foreseeable future.

• Growth Rates: Analysts estimate the global AI chip market will grow at a compound annual growth rate of 30 - 40% + over the next 3 - 5 years. The major companies producing various chips for the AI market have shared similarly aggressive estimates during quarterly earnings reports. For example, TSMC forecasts AI server demand will experience a 50% annual growth rate over the next five years, and AMD expects a 70% annual growth rate for data center chips through 2027. The transformer-specific chip market is a bit harder to estimate since it’s a specific vertical within the broader market, but it will probably outpace the general AI chip market’s growth rate because large language models like OpenAI’s GPT bots are based on a transformer architecture, and there is still an insatiable demand to apply these capabilities in a variety of applications.

• Ecosystem Dynamics: The cloud market is evolving to support more specialized chips, as Google Cloud, Amazon AWS, Microsoft Azure and other major technology companies like Apple have all been building their own distinctive chips to vertically integrate their products and reduce their reliance on Nvidia, AMD and other chip manufacturers. The future landscape appears to metamorphosing into a place where there are many chip makers who produce unique chips for distinct use cases, and Etched wants to be the best chip for transformer models.

Industry Trends 📊

The Indicators

• Catalysts: Microsoft and other major cloud providers are reportedly planning to build massive $100 billion USD data centers to power their AI demands in the cloud for the next several years. Meanwhile, Apple is leading a separate trend to build chips to run AI workloads on phones, computers and other devices, rather than have the data leave the device and run on the cloud for privacy, security and latency concerns. So on multiple fronts, the demand for tailored chip architectures is rising at an astronomical rate to meet the specific needs of both cloud and edge device computing.

• Challenges: Since Nvidia has the largest available supply of the best performing chips on the market, their ecosystem is able to strategically incentivize and systematically create lock-in effects to discourage customers from switching to other chip providers. Furthermore, since all the major cloud providers are building specialized chips of their own, the market is dominated by companies with the largest technology platforms and seemingly indomitable network effects. This is why the aforementioned comparable companies we referenced earlier — Groq, Cerebras, Lightmatter — and other ambitious startups in the space have each raised over $500 million USD just to get started. A lot of capital and resources are required to compete in this market, and Etched may not yet have enough of either.

• Recommended Reading: Chip War: The Fight for the World's Most Critical Technology by Chris Miller details decades-long conflicts between nations and businesses like within the semiconductor industry.

A Toolkit To Go 🛠️

The Equipment

• 🧰️ Code Carbon - A software package that estimates the amount of CO² produced by the cloud or personal computing resources used to execute the code.

• 🤖 MLPerf - An standard suite of benchmark tests to measure, train and infer performance of machine learning hardware, software, and services.

• 🗞️ The Aurorean Newsletter - Our team’s weekly roundup of STEM’s most significant stories of progress. We scour 100+ sources so you don’t have to.

Share your tools with us →

Share Your Thoughts 💬

The Community Wisdom

The results are in from our poll last week. Medivis is a company on a mission to improve the lives of patients by ushering in a new visual era for medicine. The following is a snapshot of the sentiments our audience felt about them.

• Bulls 56%

  • Subscriber Perspective: 'It seems like they are already well on their way to building a strong network effect with the platform they are building. I’m surprised they gained traction with so many institutions in a variety of different geographies already, because medical settings and the healthcare industry at large are known for operating slowly due to regulations, concerns about patient safety and other factors. I don’t know how much larger their customer base and partner network needs to grow until it reaches an inflection point from a financial perspective, but they must be fast approaching the mark, and I expect them to eventually get there.’

• Bears 44%

  • Subscriber Perspective: ‘The value of their mission and their traction of their product is clear. What is unclear to me is how this translates to a great financial outcome. Their pricing power should markedly decrease over time because Meta, Apple and other Big Tech companies are investing heavily in augmented reality (AR) hardware to bring general-purpose consumer products to the masses. Meanwhile, it is getting easier and cheaper to build progressively powerful software solutions with each passing day. I’m not a healthcare expert, so maybe these answers are obvious to someone else, but from my vantage point it looks like the organization will operate on thinner margins as time goes by, which means they will have less room for error if they want to succeed.’