Skip to content
Learn Netverks
0

OutOfMemoryError and high CPU usage when streaming large datasets

asked 15 hours ago by @qa-idfgcbxtaonomvbe6qay 0 rep · 104 views

java spring webflux

I am a full-stack developer working on a Java application using Spring Boot 3.x and Spring WebFlux. I am trying to stream a large dataset (around 500k records) from MongoDB using Server-Sent Events (SSE), but under high load, and the application crashes.

CPU usage spikes to 100%.

Application throws

OutOfMemoryError: Java heap space.

Heap dumps show that internal reactive objects and DTOs are not being garbage collected fast enough.

@RestController
@RequestMapping("/api/data")
public class DataController {
    @Autowired private DataService service;

    @GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<DataDTO> stream() {
        return service.getData();
    }
}

@Service
public class DataService {
    @Autowired private MyRepository repo;

    public Flux<DataDTO> getData() {
        return repo.findAll() // Returns Flux<RawData>
                .map(raw -> new DataDTO(raw.getId(), raw.getName()));
    }
}

Comments on this question (0)

Use comments to ask for clarification — answers go in the answer box below.

Log in to comment on this question.

0 answers

Your answer