I had some complex last weeks and switching 2 times the job...
This made me kind of a busy person and definitely distracted for this blog.
But I will make some Youtube clips in the dedicated page with the same name.
So, if it is your cup of tea, please follow and comment on it.
Wednesday, May 31, 2017
Thursday, January 19, 2017
How to write a logger, or how not to write one...
Loggers are bad, slow. Martin Thomson, a very known performance guy (and he is very skilled), he said: "what you see in a logger, just do the opposite".
I have seen a logger which could not write more than 1000 logs per second. In part, the reason for this slowness, it was that for every log file, a logger would open a file for append, it would write, flush and close.
A simple solution is to batch it and make it async. So, keeping the logging out of the main thread, and using a separate thread to build the logs.
In short, the writing 8 million log entries would take around 9200 ms (.Net x64). If you use .Net Core it would run in around 5750 ms. If using Java, and the same code, the code is a bit over 6 seconds.
But the surprising part is Java 9. It could do all this processing, for Java 9 could do it in less than 2 seconds (around 1.6 seconds).
So in short, if you have a lot of logs, you may consider either Java 9 or .Net Core as a target platform to generate logs quicker, but before using these platforms, make sure that the code is optimized enough upfront.
I have seen a logger which could not write more than 1000 logs per second. In part, the reason for this slowness, it was that for every log file, a logger would open a file for append, it would write, flush and close.
A simple solution is to batch it and make it async. So, keeping the logging out of the main thread, and using a separate thread to build the logs.
In short, the writing 8 million log entries would take around 9200 ms (.Net x64). If you use .Net Core it would run in around 5750 ms. If using Java, and the same code, the code is a bit over 6 seconds.
But the surprising part is Java 9. It could do all this processing, for Java 9 could do it in less than 2 seconds (around 1.6 seconds).
So in short, if you have a lot of logs, you may consider either Java 9 or .Net Core as a target platform to generate logs quicker, but before using these platforms, make sure that the code is optimized enough upfront.
Subscribe to:
Posts (Atom)