OutOfMemoryError and high CPU usage when streaming large datasets
asked 15 hours ago by @qa-idfgcbxtaonomvbe6qay 0 rep · 104 views
I am a full-stack developer working on a Java application using Spring Boot 3.x and Spring WebFlux. I am trying to stream a large dataset (around 500k records) from MongoDB using Server-Sent Events (SSE), but under high load, and the application crashes.
CPU usage spikes to 100%.
Application throws
OutOfMemoryError: Java heap space.
Heap dumps show that internal reactive objects and DTOs are not being garbage collected fast enough.
@RestController
@RequestMapping("/api/data")
public class DataController {
@Autowired private DataService service;
@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<DataDTO> stream() {
return service.getData();
}
}
@Service
public class DataService {
@Autowired private MyRepository repo;
public Flux<DataDTO> getData() {
return repo.findAll() // Returns Flux<RawData>
.map(raw -> new DataDTO(raw.getId(), raw.getName()));
}
}