Journal of Engineering Research

With the rapid development of e-commerce and financial services, real-time risk control systems have placed higher demands on data processing timeliness, consistency, and scalability. Traditional stream-batch separation architectures based on Hive, Kafka, and Flink exhibit significant bottlenecks in feature management, data updates, and query efficiency. Building on my hands-on experience in constructing big data and risk control platforms for multiple internet enterprises, this paper proposes and implements a real-time feature lake architecture based on Apache Paimon. This architecture achieves a balance between high-concurrency data writes and low-latency feature recall. Successfully deployed in risk control scenarios for global e-commerce platforms, the system supports daily processing of hundreds of billions of events, reduces feature update latency to seconds, and enhances query performance by over threefold, providing a reusable architectural paradigm for large-scale real-time risk control. The paper elaborates on system architecture design, key technology selection and optimization strategies, and provides an in-depth summary of challenges and solutions encountered during implementation.

Real-time feature lake; Risk control system; Apache Paimon; High-concurrency writes; Low-latency queries; Stream-batch integration; Feature engineering