How do you scale 4,000 microservices while tackling latency, bottlenecks, and fault tolerance? At Wix, Kafka powers our event-driven architecture with practical patterns that enhance scalability and developer velocity. This talk explores four key patterns for asynchronous programming: Integration Events: Reduce latency by pre-fetching instead of synchronous calls. Task Queue: Streamline workflows by offloading non-critical tasks. Task Scheduler: Enable precise, scalable scheduling for delayed or recurring tasks. Iterator: Handle long-running jobs in chunks for resilience and scalability. Learn how to balance benefits and trade-offs, with actionable insights to optimize your own microservices architecture using these proven patterns.

This session unveils the multifaceted horizontal scaling strategies that power Wix's robust infrastructure. From Kafka consumer scaling and dynamic traffic routing for site segments to DynamoDB sharding and MySQL clusters custom routing with ProxySQL, we dissect the mechanisms that ensure scalability and performance at Wix. Attendees will learn about the art of sharding and routing key selection across different systems, and how to apply these strategies to their own infrastructure. We'll share insights into choosing the right scaling strategy for various scenarios, balancing between managed services and custom solutions. Key Takeaways: Grasp various sharding techniques and routing strategies used at Wix. Understand key considerations for sharding key and routing rule selection. Learn when and why to choose specific horizontal scaling strategies. Gain practical knowledge for applying these strategies to achieve scalability and high availability. Join us to gain a blueprint for scaling your systems horizontally, drawing from Wix's proven practices.

Speaker : Natan Silnitsky Discover how Wix transitioned from complex event sourcing and CQRS to streamlined CRUD services, optimizing their vast platform for better scalability, performance, and resiliency. Wix's platform, designed to accommodate diverse business needs, boasts: * 3.5 Billion daily HTTP transactions * 70 Billion Kafka messages per day * Roughly 4000 microservices in production This session will highlight the simplification of Wix's architecture through domain events, resilient Kafka messaging, and advanced techniques like materialization and caching. By standardizing APIs and employing tools like protobuf and gRPC, Wix has enhanced the developer experience, both internally and externally, and fostered an open, integrative platform. Attendees will gain insights into Wix's strategies for microservice coordination, ensuring system resilience and data consistency, as well as query performance optimization through innovative 2-level caching solutions.

Workflow engines and event streaming brokers offer very different solutions to the same requirement - an optimal implementation of microservices communication. At Wix, we have a good experience with event-driven architecture for our 2500 microservices using Apache Kafka. Apache Kafka provides: - Support for very high throughput - Fault tolerance - Very loose coupling - Huge connectors eco-system Temporal workflow orchestration has interesting features: - Support for long running tasks - Business flows visual tracking - Easy to follow imperative style programming In this talk we will learn about the tradeoffs between the two technologies and how to implement various use cases in each architecture, including those that need a little more work. ABOUT CONFLUENT Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion – designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io. #confluent #apachekafka #kafka

An atomic habit is a regular practice or routine that is not only small and easy to do but is also the source of incredible power. In this talk we will learn how to become better developers by changing our habits in a way that will compound into remarkable results. We will see how to implement the Four Laws of Behavior Change (make it obvious, make it attractive, make it easy, make it satisfying) for the following habits: * getting into coding flow and avoiding distractions * learning new tech skills * prioritizing coding tasks. You can join us at the next conference! Get your ticket today! 🎟️https://bit.ly/3JJwWIK

During this session developers and architects will get many concrete takeaways about how to improve their event-driven design with proven battle-tested methods or convince them to switch from request-reply architecture. Wix has a huge scale of event driven traffic. More than 70 billion Kafka business events per day. Over the past few years Wix has made a gradual transition to an event-driven architecture for its 2000 microservices. We have made mistakes a long the way but have improved and learned a lot about how to make sure our production is still maintainable, performant and resilient. In this talk you will hear about the lessons we learned including: 1. The importance of atomic operations for databases and events 2. Avoiding full-blown event sourcing, instead taking CRUD to the next level 3. Having essential events debugging and quick-fix tools in production and a few more You can join us at the next conference! Get your ticket today! 🎟️https://bit.ly/3JJwWIK

For more info on the next Devoxx UK event 👉 www.devoxx.co.uk How do you migrate your Kafka cluster while still allowing full user traffic? Recently, Wix migrated its self-hosted, 60B events per day Kafka clusters, to a managed Kafka platform. The classic approach would be to perform this transition when all incoming traffic is removed from the data center. But draining an entire data-center for an undetermined period of time, until all 2000 services complete the switch was too risky for us. This talk is about how we gradually migrated all of our Kafka consumers and producers with 0 downtime while they continued to handle regular traffic. You will learn practical steps you can take to greatly reduce the risks and speed up the migration timeline.

Scala 3 is going to make Scala easier to write, and especially read. More power features like enums and less misleading keywords like implicit. But first, we need to migrate our old Scala 2.12 / 2.13 codebase to Scala 3. This talk tells the story of how I tried to migrate the Greyhound Kafka SDK open-source library to Scala 3 with partial success. You will hear about what works, what doesn't, and about a few pitfalls to avoid. Migration takeaways include: 1. Use migration tools, don't do it manually. 2. Which popular 3rd party libraries can and can't be used by Scala 3 code and many more. https://2022.scala.love https://twitter.com/scala_love

Fibers are the backbone of the highly performant, asynchronous, and concurrent abilities of ZIO. They are lightweight “green threads”, implemented by the ZIO runtime system. In this talk you will learn about: * How to handle fibers dying due to unexpected failures * How to guarantee a ZIO fiber is interrupted * How to set up fiber tracking and execute a fiberDump for increased debuggability. #functionalscala #FunScala2021 #Scala #zio #Ziverge #zymposium #functionalprogramming #scaladeveloper #zioscala