Ray 2.59.0: LLM APIs Stable, Token Auth Enabled by Default, and Disk-backed Shuffle
Ray 2.59.0 officially stabilizes its LLM APIs, introduces default token authentication for local clusters, delivers a new disk-backed shuffle engine, and brings several developer-facing enhancements and removals.
What changed?
Ray 2.59.0 delivers several notable infrastructure updates for Ray users: - Ray Data LLM and Ray Serve LLM APIs are now General Availability (GA)/Stable, meaning their interfaces are production-ready and supported. - Token authentication is now enabled by default for local clusters. On initialization, Ray generates and stores a token (unless explicitly disabled). - The new shuffle backend (shuffle_v2) is disk-backed and now available with Join/Aggregation support, moving beyond the in-memory approach for large workloads. - Support was added for native ORC file reading, multi-path reads in Lance, and video read imports with frame-per-second and resize parameters. - Deprecated APIs (deploy_mode/ServeDeployMode, use_new_handle_api, RAY_AGENT_ADDRESS deprecation path) are now removed. Use of '#' in deployment names now triggers a ValueError due to rep ID parsing. - Serve backpressure handling (BackpressureConfig) can now opt to return HTTP 429 (Too Many Requests) instead of 503 (Service Unavailable), enhancing API and load balancer behavior.
Why does it matter to an everyday developer?
These changes have direct developer impact: - The GA status of the Ray Data/Serve LLM APIs assures you that LLM integration points are stable and safe for production workflows. - Token authentication improves local security with minimal developer effort; local Ray clusters are harder to access undesirably, and token management is handled automatically. This does not affect remote or multi-node clusters yet. - Disk-backed shuffle (shuffle_v2) removes memory-based limits for large aggregations and joins, improving scalability when dealing with large datasets. - The new backpressure configuration in Ray Serve enables more correct error codes for HTTP consumers and improves client-side retry or backoff strategies. - Removal of deprecated APIs means legacy code relying on old options may now break, so code maintenance may be needed.
