Massive consequential output hangs the server

[ivitlin]

Hello!

Really appreciate your work, great tool!
The only problem I've encountered is that consequential output (like wget download progress bar or pip installation bar and similar) isn't supported to be displayed on single line as it's intended and supported by terminal apps like Putty.
For example:
In Putty, the bar is filled on the same line:
Downloading ***.BIN.ZIP ...
|========================>| Got 130480561 bytes of 130480561 (100.00%)

And in terminal of script-server it looks like this:
Downloading ***.BIN.ZIP ...
|> | Got 864 bytes of 130480561 (0.00%)
|> | Got 4960 bytes of 130480561 (0.00%)
|> | Got 5208 bytes of 130480561 (0.00%)
|> | Got 8104 bytes of 130480561 (0.01%)
|> | Got 9552 bytes of 130480561 (0.01%)
|> | Got 11000 bytes of 130480561 (0.01%)
|> | Got 12448 bytes of 130480561 (0.01%)
|> | Got 13896 bytes of 130480561 (0.01%)
|> | Got 16792 bytes of 130480561 (0.01%)
|> | Got 18240 bytes of 130480561 (0.01%)
|> | Got 19688 bytes of 130480561 (0.02%)
|> | Got 21136 bytes of 130480561 (0.02%)
|> | Got 22584 bytes of 130480561 (0.02%)
|> | Got 24032 bytes of 130480561 (0.02%)
|> | Got 25480 bytes of 130480561 (0.02%)
..............................
This behavior seems to overflow the server, CPU reaches 100% and the server becomes unresponsive and needs to be restarted.
Can this behavior be fixed?

Thanks!

[bugy]

Hi @ivitlin,
Thank you for the report! I'll try to have a look later today.
The problem with single line output is clear (this behaviour is not supported yet)

But the problem with CPU looks strange to me. According to your log, there should be ~50000 lines of text in total, which should be fine for script-server, especially without bash formatting.
May I ask you for some help with this investigation? Could you try to add and run my sample script with printing a file? https://github.com/bugy/script-server/blob/master/samples/configs/write_file.json and https://github.com/bugy/script-server/blob/master/samples/scripts/my_script.sh
And print some text with 50000 repeats there? Usually for me it takes ~1 minute for a browser to render, but server finishes quite fast

In any case, I'll try to reproduce the issue with wget myself also

[ivitlin]

Hi, I did as you said, even 100000 lines were generated successfully, both in file and in terminal. I think it's not the number of lines what's hangs the server, but frequency of terminal update requests. Unfortunately, I can't provide scenario when the issue reproduces in 100% cases. Sometimes server seems to be hanged, but after 3-5 mins it's back to life. Besides, I'm running the server on weak VM, it can "contribute" to the issue too. Anyway, do you plan to add support for single line output? If you do, can we open enhancement request?

[bugy]

Hi @ivitlin, thank you for providing the test results! One more thing came to my mind - performance problem could be due to not effective bash formatting in case of unsupported formatting code (replace line).
Could you try to disable bash formatting for this script config please? This can be done with the following parameter
"bash_formatting": false

Regarding support for single line output - I'll implement it within the scope of this task. Probably within a week

[ivitlin]

Great workaround - disabling bash formatting really helped: CPU load didn't even reach 30% and script finished running in 30 sec, while the same script with bash formatting enabled took 100% of CPU and run for 3 minutes. Many thanks! Anxiously waiting for support for single line output.

[bugy]

Thanks for the feedback, I'm glad this setting helped!
I will try provide single line support as soon as possible

[bugy]

Hi @ivitlin, could you tell me, please, which tool are you using for download? I checked wget, but I get different output for it and no performance issues. Most probably the tool you used is using different codes for formatting and I would like to add support for both (wget style and the one you have).

[ivitlin]

Hi, the output if from Perl-based IP2Location download script:
https://www.ip2location.com/free/downloader

[bugy]

Hi @ivitlin, thank you. Unfortunately it seems that it's not available for free testing. May I ask you to send me a partial execution .log file from logs/processes folder (or a full one, if there is nothing private) ? In this file exact output is logged, so it should be enough for me
You can attach it here or email to me: buggygm@gmail.com

[ivitlin]

Hi, sent to mail

[bugy]

Hi @ivitlin, thank you for sharing the file!

According to .pl file, \r character is used for replacing the last line.
However in the log file I see only \n characters. It could be another bug, that \r characters are ignored there...

Anyway, it seems that most download tools are using \r for amending the last line. So I'll work on this case at first and then add other formatting support.
At the moment I'm working at the implementation

[bugy]

Just an update: I'm still working on this issue, unfortunately it took a bit longer than expected. New estimate is to finish until the end of the week.

[bugy]

Done. The fixes and improvements are now in master:

• added support for \r (carriage return), now text following \r will replace begginning of the line
• fixed logging, now logs will preserve \r characters
• in PTY mode (default on linux), all new lines were replaced to \r\n. I disabled this, so \r and\or \n are outputted as they are

@ivitlin unfortunately I couldn't find the reason for performance issues. Could you check please, if they will stay after all changes?