improve unittests and fix strptime of status datetime for 3.6 #145

Vitalts · 2021-10-12T10:56:19Z

• mock all requests to real overpass endpoint in unittests
• refactor unittests
• add more unittests
• fix strptime of status datetime for 3.6

…he "update_examples" function with a start by the value of global variable "UPDATE_EXAMPLES"

mvexel · 2021-11-18T19:16:06Z

Hi @Vitalts - after merging @dericke 's PR #141 there are some minor merge conflicts, could you resolve and then I can merge, please?

dericke · 2021-11-19T18:13:45Z

We've dropped py3.6 support since this PR was opened, so the parts related to that shouldn't be merged.

mvexel · 2022-03-22T18:06:27Z

@Vitalts Are you still interested in getting this merged? See also @dericke's comment. Sorry for the slow turnaround.. Let me know if I can help in any way.

mvexel · 2024-01-08T00:11:34Z

@dericke @Vitalts how would you like to proceed with this, in light of the significant changes in test_api.py in #140 ?

dericke · 2024-01-08T21:21:07Z

I won't be able to spend any time on this for a couple of weeks, but I'll check in and see how I can help out at that point.

dericke · 2024-01-23T19:16:31Z

overpass/api.py

@@ -93,7 +95,7 @@ def get(self, query, responseformat="geojson", verbosity="body", build=True, dat
                date = datetime.fromisoformat(date)
            except ValueError:
                # The 'Z' in a standard overpass date will throw fromisoformat() off
-                date = datetime.strptime(date, '%Y-%m-%dT%H:%M:%SZ')
+                date = self._strptime(date)


This change is not needed since we've dropped 3.6 support

dericke · 2024-01-23T19:18:44Z

overpass/dependency.py

@@ -0,0 +1,9 @@
+import sys


Deprecated by the dropping of 3.6

dericke · 2024-01-23T19:24:48Z

overpass/api.py

+    def _strptime(date_string):
+        if dependency.Python.less_3_7():
+            dt = datetime.strptime(date_string, '%Y-%m-%dT%H:%M:%SZ')
+            kwargs = {k: getattr(dt, k) for k in ('year', 'month', 'day', 'hour', 'minute', 'second', 'microsecond')}
+            kwargs['tzinfo'] = timezone.utc
+            return datetime(**kwargs)
+
+        return datetime.strptime(date_string, '%Y-%m-%dT%H:%M:%S%z')
+
+    @classmethod
+    def _api_status(cls) -> dict:


Also deprecated by the dropping of 3.6 support

you mean we can get rid of the custom _strptime() function altogether and replace with plain datetime.strptime()?

dericke · 2024-01-23T19:25:30Z

overpass/api.py

+    def __deprecation_get(self, *args, **kwargs):
+        import warnings
+        warnings.warn('Call to deprecated function "Get", use "get" function instead', DeprecationWarning)
+        return self.get(*args, **kwargs)
+
+    def __deprecation_search(self, *args, **kwargs):
+        import warnings
+        warnings.warn('Call to deprecated function "Search", use "search" function instead', DeprecationWarning)
+        return self.search(*args, **kwargs)
+
    # deprecation of upper case functions
-    Get = get
-    Search = search
+    Get = __deprecation_get
+    Search = __deprecation_search


Still relevant with the dropping of 3.6.

dericke · 2024-01-23T19:25:43Z

overpass/api.py

+        waiting_slots = tuple(cls._strptime(waiting_re.search(line).group())
+                              for line in lines if waiting_re.search(line))

        current_idx = next(
            i for i, word in enumerate(lines)
            if word.startswith('Currently running queries')
        )
        running_slots = tuple(tuple(line.split()) for line in lines[current_idx + 1:])
-        running_slots_datetimes = tuple(
-            datetime.strptime(
-                slot[3], "%Y-%m-%dT%H:%M:%S%z"
-            )
-            for slot in running_slots
-        )
+        running_slots_datetimes = tuple(cls._strptime(slot[3]) for slot in running_slots)


Also deprecated by the dropping of 3.6 support

dericke · 2024-01-23T19:26:03Z

tests/__init__.py

+RESOURCE_PATH = os.path.join(os.path.dirname(os.path.abspath(os.path.relpath(__file__))), 'resources')
+
+
+def save_resource(file_name, data):
+    with open(os.path.join(RESOURCE_PATH, file_name), 'wb') as f:
+        f.write(data)
+
+
+def load_resource(file_name):
+    with open(os.path.join(RESOURCE_PATH, file_name), 'rb') as f:


Can we use pathlib instead of os.path?

Not familiar. What does that give us?

dericke · 2024-01-23T19:27:29Z

Everything in the files I didn't comment on looks to still be relevant at first glance (and great work there!) but I haven't run any of the tests yet to be sure.

mvexel · 2024-01-29T00:17:01Z

@Vitalts are you still interested in getting this over the finish line? I'm sorry it all took so long.. With @dericke 's comments we are one step closer.

also migrated some metadata to pyproject.toml

mvexel · 2024-02-12T19:29:35Z

so @dericke @Vitalts This branch has diverged from main enough that merging will be a bit of a mess. I could use some help especially with api.py

Anyone has some time to spend on this? Or should we abandon this PR and start a new one focused on fixing the tests and deprecating 3.6?

dericke · 2024-02-19T15:31:16Z

I would be in favor of closing this PR and opening a new one focused just on the tests.

mvexel · 2024-02-21T17:39:38Z

thanks @dericke , I will close this then and create an issue that we can create a new PR against.

Vitali Balandin added 14 commits September 3, 2021 17:03

refactor, collect all status tests in the TestSlots class

fd2da86

refactor, parametrize test_geojson

b40baa9

refactor, mock requests

7c88566

refactor, move example sources

1b4ceac

add unittest test_overpass_id

20139f5

add queries unittests

7a1bd56

add api errors unittests

7208cec

refactor, move the commented code for updating "example.*" files to t…

56885a7

…he "update_examples" function with a start by the value of global variable "UPDATE_EXAMPLES"

add deprecation warning messages to deprecated functions

883f49a

fix strptime of status datetime

6bfac3a

use 3.7 future for strptime of status datetime

4396fad

refactor, move Python version check to dependency.py

ef18a2c

refactor, make "_strptime" a static method

2e4b234

use static "_strptime" method to parse "date" argument of "get" method

55ab07c

dericke suggested changes Jan 23, 2024

View reviewed changes

fix tests, deprecate 3.6 related functions

0f376ec

also migrated some metadata to pyproject.toml

mvexel closed this Feb 21, 2024

improve unittests and fix strptime of status datetime for 3.6 #145

improve unittests and fix strptime of status datetime for 3.6 #145

Uh oh!

Conversation

Vitalts commented Oct 12, 2021

Uh oh!

mvexel commented Nov 18, 2021

Uh oh!

dericke commented Nov 19, 2021

Uh oh!

mvexel commented Mar 22, 2022

Uh oh!

mvexel commented Jan 8, 2024

Uh oh!

dericke commented Jan 8, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dericke commented Jan 23, 2024

Uh oh!

mvexel commented Jan 29, 2024

Uh oh!

mvexel commented Feb 12, 2024

Uh oh!

dericke commented Feb 19, 2024

Uh oh!

mvexel commented Feb 21, 2024

Uh oh!

Uh oh!