Wagtail has shared the results of a one-month experiment conducted throughout September to use the GLM 5.3 Flash model for day-to-day engineering tasks. While the goal was to rely solely on this efficient model, the experiment resulted in only 50% of the total 2 billion tokens being attributed to the target model, with the remaining 1 billion tokens being redirected to other models.
Wagtail reports unexpected costs and resource challenges during one-month GLM 5.3 Flash experiment
The experiment faced significant hurdles, including high resource consumption and infrastructure limitations. Wagtail reported that a prototype of their experimental MCP (Model Context Protocol) server consumed approximately 450 million tokens, costing $150 and 5 kWh of energy almost overnight. Furthermore, the team encountered performance degradation in GLM 5.3 Flash, likely due to high demand and limited capacity compared to larger AI labs. These availability issues forced the team to switch to alternative models, such as DeepSeek V4.1 Flash and Qwen 3.8 Flash.
Despite these challenges, Wagtail noted that the experiment provided crucial data for benchmarking model performance on Wagtail tasks. The team concluded that while focusing on flash-tier models is viable for developer workflows, careful model selection and budgeting for agentic patterns are essential to avoid unexpected costs and energy usage.
Sources
- One month coding with GLM 5.3 Flash （Hacker News Frontpage, 2026-10-02）
- Torchbox