How do you migrate your Kafka cluster while still allowing full user traffic? Recently, Wix migrated its self-hosted, 60B events per day Kafka clusters, to a managed Kafka platform. The classic approach would be to perform this transition when all incoming traffic is removed from the data center. But draining an entire data-center for an undetermined period of time, until all 2000 services complete the switch was too risky for us. This talk is about how we gradually migrated all of our Kafka consumers and producers with 0 downtime while they continued to handle regular traffic. You will learn practical steps you can take to greatly reduce the risks and speed up the migration timeline.

Scala 3 is going to make Scala easier to write, and especially read. More power features like enums and less misleading keywords like implicit. But first, we need to migrate our old Scala 2.12 / 2.13 codebase to Scala 3. This talk tells the story of how I tried to migrate the Greyhound Kafka SDK open-source library to Scala 3 with partial success. You will hear about what works, what doesn't, and about a few pitfalls to avoid. Migration takeaways include: 1. Use migration tools, don't do it manually. 2. Which popular 3rd party libraries can and can't be used by Scala 3 code and many more. https://2022.scala.love https://twitter.com/scala_love

Fibers are the backbone of the highly performant, asynchronous, and concurrent abilities of ZIO. They are lightweight “green threads”, implemented by the ZIO runtime system. In this talk you will learn about: * How to handle fibers dying due to unexpected failures * How to guarantee a ZIO fiber is interrupted * How to set up fiber tracking and execute a fiberDump for increased debuggability. #functionalscala #FunScala2021 #Scala #zio #Ziverge #zymposium #functionalprogramming #scaladeveloper #zioscala

About a year ago data streams team at Wix has released to open-source its Kafka client SDK wrapper called Greyhound. Greyhound offers rich functionality like message processing parallelisation and batching, various fault tolerant retry policies and much more. This talk will show how the team designed Greyhound with a layered architecture to allow both public and private parts and also different levels of flexible configuration. How it automatically syncs only relevant code from private repo to public one and also how it securely accepts public PRs back to the private repo. From https://summit2021.reversim.com/

The talk from Devoxx Ukraine 2021 ONLINE Fb: www.facebook.com/DevoxxUkraine/ Website: www.devoxx.com.ua Architecture Wix has a huge scale of traffic. more than 500 billion HTTP requests and more than 1.5 billion Kafka business events per day. This talk goes through 3 Caching Patterns that are used by Wix's 2000 microservices in order to provide the best experience for its users along with saving costs, reducing latency and increasing availability. The patterns include: * Configuration Data Cache - persisted locally or to S3 * HTTP Reverse Proxy Caching - using Varnish Cache * (Dynamo)DB+CDC based Cache - for unlimited capacity with continuously updating LRU cache on top each pattern is optimal for other use cases, but all allow to reduce costs and gain performance and resilience.

As your organization rapidly grows in scale, so does the number of challenges you face. The growing scale comes in multiple dimensions - traffic, geographic presence, products portfolio, various technologies, number of developers, etc. Coming up with an architecture that can handle all of the data flows in a universal, simple way is key. This talk is about Wix's Kafka-based global data architecture and platform. How we made it very easy for 2,000 Wix microservices to publish and subscribe to data, no matter where they are deployed in the world, or what technology stack they use. All the while offering various tools and features to help adapt to the growing scale and insuring high resilience. // bio Natan Silnitsky is a backend-infra team lead @Wix.com. He leads the Data streaming team in charge of building event-driven libraries and tools on top of Kafka and ZIO. Before that, he was part of a task force that was responsible for building the next-generation CI system at Wix on top of Google's Bazel build tool. Has many years of experience as a developer of large-scale web services - First in .Net, later in Scala. Natan's passions include clean and functional code, dev velocity, and great software design.

You want to introduce ZIO to your existing Scala codebase? Great Idea! It will make your code more efficient, readable, composable, and safe. For the past year, Natan Silnitsky has done this at Wix and has learned a lot about how to do it right. In this talk, Natan will show you how to successfully use ZIO in your legacy service using real-life code examples. You will learn key tips and takeaways including how and when to execute the ZIO runtime; how/when to introduce ZLayers into your codebase; how to make your existing APIs interop with ZIO; and how to have more flexibility on ZManaged resource shutdown. When you're done attending the presentation, you'll be able to introduce ZIO into your existing Scala code base with confidence! About Functional Scala: Functional Scala was founded in 2019 to provide an independent, professional platform for speakers across the entire Scala community, bringing together new and existing Scala developers in an uplifting and welcoming environment that focuses on innovation & excellence. Join the Functional Scala Community on Twitter: https://twitter.com/funscala2021 and https://twitter.com/zivergetech Stay informed on future events: https://www.functionalscala.com/, https://ziverge.com/ Join us in 2021: https://www.eventbrite.co.uk/e/functional-scala-2021-tickets-131620665955 #FunctionalScala2020 #FunScala2020 #FunScala20

The talk from Devoxx Ukraine 2020 ONLINE Fb: www.facebook.com/DevoxxUkraine/ Website: www.devoxx.com.ua During the past couple of years I’ve implemented or have witnessed implementations of several key patterns of event-driven messaging designs on top of Kafka that have facilitated creating a robust distributed system at Wix that can easily handle increasing traffic and storage needs with many different use-cases. In this talk I will share these patterns with you, including: * Consume and Project (data decoupling) * End-to-end Events (Kafka+websockets) * In memory KV stores (consume and query with 0-latency) * Events transactions (Exactly Once Delivery) #devoxxua #java #devoxx

The talk from Devoxx Ukraine 2020 ONLINE Fb: www.facebook.com/DevoxxUkraine/ Website: www.devoxx.com.ua At Wix, we have created a universal event-driven programming infrastructure on top of the Kafka message broker. This infra makes sure messages are eventually successfully consumed and produced no matter what failure it encounters. In this talk, you will learn about the features we introduced in order to make sure our distributed system can safely handle an ever growing message throughput in a fault tolerant manner. You will be introduced to such techniques as retry topics, local persistent queues, and cooperative fibers that help make your flows more resilient and performant. You will also learn how to make this infra work for all programming languages tech stacks with optimal resource manage using the power of Kubernetes and gRPC. When to use a client library, and when to deploy an external pod (DaemonSet, StatefulSet) or even deploy a sidecar. #devoxxua #java #devoxx