Skip to content

refactor nettools and add tests #531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 120 commits into from
Mar 11, 2021
Merged

Conversation

sameh-farouk
Copy link
Member

@sameh-farouk sameh-farouk commented Jan 3, 2021

Description

Resolves #526

Changes

  • refactoring the code, fixing bugs, adding missing error handlers
  • Adding proper logging
  • reimplementing many functions
  • ensure complete docstrings and proper code format
  • adding unit tests
  • adding compatibility for IPv6
  • ensure consistent behavior on both Mac OSX and Linux operating systems
  • new get_free_port function to help in picking granted free and ready-to-use ports

all commits: https://github.com/threefoldtech/js-ng/pull/531/commits

Known issues:

  • get_network_info execute the ip command on *nix platforms. it raises NotImplementedError on other platforms.

Related Issues

Resolves #526
Resolves #569
related to #427

Checklist

  • Pre-commit hook is installed to do formatting checks before committing code...etc
  • Tests included
  • Build pass
  • Documentation
  • Code format and docstrings

Using socket.create_connection instead of socket.connect
This allow the function to work for both IPv4 and IPv6 addresses vs
the current current implementation that support only IPv4 (AF_INET).
also if host is a non-numeric hostname, it will try to resolve it for both
AF_INET and AF_INET6,
and then try to connect to all possible addresses in turn until a
connection succeeds.
- Respecting the timeout arg passed to nettools.wait_http_test() function
- Add a customized interval_time with a default value 2
- update the Docstring
- adding arg to allow fake the user-agent, default to True.
- use standard library, avoid unnecessary dependencies and make the code
  more explicit.
- add proper errors handlers.
- update the function docstring.
- add compatibility for IPv6.
- update the docstring to be more clearer.
- add a default value for port arg as it does not matter much to
  simplifying the function call.
- Add compatibility to IPv6 in nettools.get_default_ip_config()
- parameterizing the hard coded value for the ip address used to get the
  puplic facing ip
- make use of Json instead of depending or regex parsing.
- optimize the code to get only the device that matter when pass 'device'
  arg to the function, instead of looping over all device.
- depending on one shell command instead of multiple commands.
- handle the error in cases when OS not supported.
- update docstring.
- now return cidr for both ipv4 and ipv6. 'ip', 'ip6' dict keys conatin
  list of (ip, prefix) pairs.
- update of other nettools functions that use old version output.
  get_default_ip_config() and _netobject_get()
- getting more acurate timeout by passing the timeout value to ping command
  instead of implementing it with a while loop
- handle the errors when running on unsupported os
@codecov-io
Copy link

codecov-io commented Jan 3, 2021

Codecov Report

Merging #531 (38fbde1) into development (240e5ff) will increase coverage by 2.36%.
The diff coverage is 73.58%.

Impacted file tree graph

@@               Coverage Diff               @@
##           development     #531      +/-   ##
===============================================
+ Coverage        68.31%   70.68%   +2.36%     
===============================================
  Files               91       91              
  Lines             4561     4608      +47     
===============================================
+ Hits              3116     3257     +141     
+ Misses            1445     1351      -94     
Impacted Files Coverage Δ
jumpscale/sals/nettools/__init__.py 64.38% <73.58%> (+43.17%) ⬆️
jumpscale/sals/process/__init__.py 62.81% <0.00%> (+0.36%) ⬆️
jumpscale/sals/fs/__init__.py 63.22% <0.00%> (+0.41%) ⬆️
jumpscale/core/base/factory.py 92.67% <0.00%> (+0.52%) ⬆️
jumpscale/core/base/meta.py 92.96% <0.00%> (+0.78%) ⬆️
jumpscale/tools/startupcmd/startupcmd.py 86.79% <0.00%> (+1.25%) ⬆️
jumpscale/servers/gedis/server.py 65.26% <0.00%> (+2.63%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 240e5ff...38fbde1. Read the comment docs.

this is necessary to fix repeated test docs #553
Copy link
Contributor

@abom abom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, we should do less logging in general (e.g. exceptions only)

@sameh-farouk
Copy link
Member Author

IMHO, we should do less logging in general (e.g. exceptions only)

could we leave it for now? it will be useful in this stage in debugging any connection issues, later we can make it less and benefit from the performance gain.

what do you think?

@sameh-farouk sameh-farouk deleted the sameh-farouk-iss526-nettools branch March 1, 2021 18:04
@xmonader xmonader restored the sameh-farouk-iss526-nettools branch March 1, 2021 18:17
@xmonader xmonader reopened this Mar 1, 2021
@abom abom mentioned this pull request Mar 7, 2021
5 tasks
@abom abom changed the title issue526 nettools refactor nettools and related tests Mar 7, 2021
@abom abom changed the title refactor nettools and related tests refactor nettools and add tests Mar 7, 2021
@abom abom merged commit 639a866 into development Mar 11, 2021
@abom abom deleted the sameh-farouk-iss526-nettools branch March 11, 2021 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

wait_http_test performs the http check timeout times with 1 second in between revisit nettools sal
6 participants