How to perform asynchronous LLM inference on Kafka streams using Apache Spark, and handle high-throughput RAG ingestion?

apache-spark asynchronous kafka-consumer-api large-language-model rag