Memory issue with getters since node 0.9.4
Problem
Hi, we was fighting with high rss consumption last days since we switched to node 0.10.12. We finally found out where the problem actually is. Its probably not directly a mongoose issue, but I think mongoose users might want to know about this and if we get a mongoose free reproducible code, then we can create an issue on node or v8 tracker. We tracked down the issue you can reproduce with the code from gist to this particular part: [code block] Behaviour When you start to send requests, memory usage gets higher and higher until OS limit is reached, gc is not able to cleanup in this specific case provided in the example. Hotfix If you use gc_global option for v8 "node --gc_global mongoose-test.js" which will reenable periodical garbage collector, the issue will not be reproducible any more. Since when https://github.com/joyent/node/commit/d607d856afc744b1e6bc42cdb9b67b6c4e7e4876 This commit is the point where the issue comes in play. In my opinion incremental gc has a bug. This commit is relased with node 0.9.4, the issue is not reproducible in all versions below and it is reproducible in all version higher. How to reproduce 1. Download 2 files from this gist https://gist.github.com/kof/5884355 2. start mongod 3. $ node mongoose-test.js 4. $ for i in {1..500}; do curl http://localhost:8888/; done
Unverified for your environment
Select your OS to check compatibility.
1 Fix
Implement Global Garbage Collection for Mongoose Tests
The issue arises from a change in V8's garbage collection strategy introduced in Node.js version 0.9.4, which affects memory management during high request loads. The incremental garbage collector fails to reclaim memory effectively under certain conditions, leading to increased RSS (Resident Set Size) until the OS limit is reached. This behavior is exacerbated when using Mongoose with MongoDB, as it may create numerous objects that are not collected promptly.
Awaiting Verification
Be the first to verify this fix
- 1
Modify Node.js Execution Command
Start your Node.js application with the global garbage collection option to mitigate memory issues. This will enable periodic garbage collection, which helps in managing memory more effectively during high load scenarios.
bashnode --gc_global mongoose-test.js - 2
Monitor Memory Usage
Use a monitoring tool or command to observe the memory usage of your Node.js application while it processes requests. This will help you confirm that the memory consumption stabilizes with the global garbage collection enabled.
bashps -o pid,rss,command -p $(pgrep -f mongoose-test.js) - 3
Run Load Tests
Execute the load test again by sending multiple requests to your application. Ensure that the same number of requests (500) is sent as before to compare memory usage before and after the fix.
bashfor i in {1..500}; do curl http://localhost:8888/; done - 4
Check for Memory Leaks
Utilize a memory profiling tool (like Chrome DevTools or Node.js built-in profiler) to check for any potential memory leaks or uncollected objects during the load test. This will help ensure that the fix is effective and that no additional issues arise.
bashnode --inspect mongoose-test.js - 5
Document Findings
Record the results of your tests, including memory usage before and after implementing the global garbage collection option. This documentation can be useful for future reference and for reporting the issue to Node.js or V8 maintainers if necessary.
bashecho 'Memory usage before and after fix' > memory_report.txt
Validation
Confirm that memory usage does not continue to increase uncontrollably during the load test and that the garbage collector is able to reclaim memory effectively. Compare the memory usage metrics before and after applying the fix to ensure a significant improvement.
Sign in to verify this fix
Environment
Submitted by
Alex Chen
2450 rep