NewsDialy
Every time a chatbot answers a question, it runs a process called inference: the step where an AI model takes input and produces output.
Advanced Micro Devices (Nasdaq: AMD) announced on August 6 that it has agreed to buy Taalas, a chip startup building hardware designed specifically for that workload.
What Taalas actually does Taalas takes a different approach from general-purpose chips. Instead of running AI models on flexible graphics processors, the company hardwires entire AI models directly into silicon.
Its HC1 demonstrator reportedly delivers around 17,000 tokens per second per user when running the Llama 3.1 8B model.
Keep reading