New examples for PyOpenPose (CMU-Perceptual-Computing-Lab#1039)

Author: soulslicer

doc/modules/python_module.md

@@ -1,5 +1,4 @@
-OpenPose Python Module and Demo
-=============================================
+# OpenPose Python Module and Demo
 
 ## Contents
 1. [Introduction](#introduction)
@@ -18,7 +17,29 @@ This module exposes a Python API for OpenPose. It is effectively a wrapper that
 ## Compatibility
 The OpenPose Python module is compatible with both Python 2 and Python 3. In addition, it will also run in all OpenPose compatible operating systems. It uses [Pybind11](https://github.com/pybind/pybind11) for mapping between C++ and Python datatypes.
 
-To compile, enable `BUILD_PYTHON` in cmake. Pybind selects the latest version of Python by default (Python 3). To use Python 2, change `PYTHON_EXECUTABLE` and `PYTHON_LIBRARY` flags in cmake to your desired python version.
+To compile, enable `BUILD_PYTHON` in cmake. Pybind selects the latest version of Python by default (Python 3). To use Python 2, change `PYTHON_EXECUTABLE` and `PYTHON_LIBRARY` flags in cmake-gui to your desired python version. 
+
+Ubuntu Eg:
+
+```
+PYTHON_EXECUTABLE=/usr/bin/python2.7 
+PYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython2.7m.so
+```
+
+Mac OSX Eg:
+
+```
+PYTHON_EXECUTABLE=/usr/local/bin/python2.7 
+PYTHON_LIBRARY=/usr/local/opt/python/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7m.dylib 
+```
+
+Windows Eg:
+
+```
+PYTHON_EXECUTABLE=C:/Users/user/AppData/Local/Programs/Python/Python27/python.exe
+```
+
+If run via the command line, you may need to run cmake twice in order for this change to take effect. 
 
 
 
@@ -61,5 +82,69 @@ python3 2_whole_body_from_image.py
 ## Exporting Python OpenPose
 Note: This step is only required if you are moving the `*.py` files outside their original location, or writting new `*.py` scripts outside `build/examples/tutorial_api_python`.
 
-- Option a, installing OpenPose: On an Ubuntu or OSX based system, you could install OpenPose by running `sudo make install`, you could then set the OpenPose path in your python scripts to the OpenPose installation path (default: `/usr/local/python`) and start using OpenPose at any location. Take a look at `build/examples/tutorial_pose/1_body_from_image.py` for an example.
-- Option b, not installing OpenPose: To move the OpenPose Python API demos to a different folder, ensure that the line `sys.path.append('{OpenPose_path}/python')` is properly set in your `*.py` files, where `{OpenPose_path}` points to your build folder of OpenPose. Take a look at `build/examples/tutorial_pose/1_body_from_image.py` for an example.
+Ubuntu/OSX:
+
+- Option a, installing OpenPose: On an Ubuntu or OSX based system, you could install OpenPose by running `sudo make install`, you could then set the OpenPose path in your python scripts to the OpenPose installation path (default: `/usr/local/python`) and start using OpenPose at any location. Take a look at `build/examples/tutorial_api_python/1_body_from_image.py` for an example.
+- Option b, not installing OpenPose: To move the OpenPose Python API demos to a different folder, ensure that the line `sys.path.append('{OpenPose_path}/python')` is properly set in your `*.py` files, where `{OpenPose_path}` points to your build folder of OpenPose. Take a look at `build/examples/tutorial_api_python/1_body_from_image.py` for an example.
+
+Windows:
+
+- Ensure that the folder  `build/x{86/64}/Release`and `build/bin` are copied along with `build/python` As noted in the example, the path for these can be changed in the following two variables:
+
+  ```
+  sys.path.append(dir_path + '/../../python/openpose/Release);
+  os.environ['PATH']  = os.environ['PATH'] + ';' + dir_path + '/../../{x86/x64}/Release;' +  dir_path + '/../../bin;'
+  ```
+
+  
+
+#### Common Issues
+The error in general is that openpose cannot be found. Ensure first that `BUILD_PYTHON` flag is set to ON. If the error persists, check the following:
+
+In the script you are running, check for the following line, and run the following command in the same location as where the file is
+
+**Ubuntu/OSX:**
+
+`sys.path.append('../../python');`
+
+```
+ls ../../python/openpose
+```
+
+Check the contents of this location. It should contain one of the following files:
+
+```
+pyopenpose.cpython-35m-x86_64-linux-gnu.so
+pyopenpose.so
+```
+
+If you do not have any one of those, you may not have compiled openpose successfully, or you may be running the examples, not from the build folder but the source folder. If you have the first one, you have compiled pyopenpose for python 3, and have to run the scripts with python3, and vice versa for the 2nd one. Follow the testing examples above for exact commands.
+
+**Windows:**
+
+Python for Openpose needs to be compiled in Release mode for now. This can be done in [Visual Studio](https://cdn.stereolabs.com/docs/getting-started/images/release_mode.png). Once that is done check this line:
+
+`sys.path.append(dir_path + '/../../python/openpose/Release');`
+
+```
+dir ../../python/openpose/Release
+```
+
+Check the contents of this location. It should contain one of the following files:
+
+```
+pyopenpose.cp36-win_amd64.pyd
+pyopenpose.pyd
+```
+
+If such a folder does not exist, you need to compile in Release mode as seen above. If you have the first one, you have compiled pyopenpose for python 3, and have to run the scripts with python3, and vice versa for the 2nd one. Follow the testing examples above for exact commands. If that still does not work, check this line:
+
+`os.environ['PATH']  = os.environ['PATH'] + ';' + dir_path + '/../../x64/Release;' +  dir_path + '/../../bin;'`
+
+```
+dir ../../x64/Release
+dir ../../bin
+```
+
+Ensure that both of these paths exist, as pyopenpose needs to reference those libraries. If they don't exist, change the path so that they point to the correct location in your build folder
+

examples/tutorial_api_cpp/08_heatmaps_from_image.cpp

@@ -43,7 +43,7 @@ bool display(
             const cv::Mat inputNetDataR(
                 height, width, CV_32F, &inputNetData.getPtr()[2*height*width]);
             cv::Mat netInputImage;
-            cv::merge({inputNetDataB, inputNetDataG, inputNetDataR}, netInputImage);
+            cv::merge(std::vector<cv::Mat>{inputNetDataB, inputNetDataG, inputNetDataR}, netInputImage);
             netInputImage = (netInputImage+0.5)*255;
             // Turn into uint8 cv::Mat
             cv::Mat netInputImageUint8;

examples/tutorial_api_python/1_body_from_image.py

@@ -63,6 +63,5 @@
 
 # Display Image
 print("Body keypoints: \n" + str(datum.poseKeypoints))
-while 1:
-    cv2.imshow("win", datum.cvOutputData)
-    cv2.waitKey(15)
+cv2.imshow("OpenPose 1.4.0 - Tutorial Python API", datum.cvOutputData)
+cv2.waitKey(0)

examples/tutorial_api_python/2_whole_body_from_image.py

@@ -68,6 +68,5 @@
 print("Face keypoints: \n" + str(datum.faceKeypoints))
 print("Left hand keypoints: \n" + str(datum.handKeypoints[0]))
 print("Right hand keypoints: \n" + str(datum.handKeypoints[1]))
-while 1:
-    cv2.imshow("win", datum.cvOutputData)
-    cv2.waitKey(15)
+cv2.imshow("OpenPose 1.4.0 - Tutorial Python API", datum.cvOutputData)
+cv2.waitKey(0)

examples/tutorial_api_python/3_heatmaps_from_image.py

@@ -35,7 +35,9 @@
 params = dict()
 params["model_folder"] = "../../../models/"
 params["heatmaps_add_parts"] = True
+params["heatmaps_add_bkg"] = True
 params["heatmaps_add_PAFs"] = True
+params["heatmaps_scale"] = 2
 
 # Add others in path?
 for i in range(0, len(args[1])):
@@ -69,7 +71,7 @@
 outputImageF = cv2.merge([outputImageF[0,:,:], outputImageF[1,:,:], outputImageF[2,:,:]])
 outputImageF = (outputImageF*255.).astype(dtype='uint8')
 heatmaps = datum.poseHeatMaps.copy()
-heatmaps = (heatmaps*255.).astype(dtype='uint8')
+heatmaps = (heatmaps).astype(dtype='uint8')
 
 # Display Image
 counter = 0
@@ -78,7 +80,9 @@
     heatmap = heatmaps[counter, :, :].copy()
     heatmap = cv2.applyColorMap(heatmap, cv2.COLORMAP_JET)
     combined = cv2.addWeighted(outputImageF, 0.5, heatmap, 0.5, 0)
-    cv2.imshow("win", combined)
-    cv2.waitKey(-1)
+    cv2.imshow("OpenPose 1.4.0 - Tutorial Python API", combined)
+    key = cv2.waitKey(-1)
+    if key == 27:
+        break
     counter += 1
     counter = counter % num_maps

examples/tutorial_api_python/4_keypoints_from_images.py

@@ -0,0 +1,79 @@
+# From Python
+# It requires OpenCV installed for Python
+import sys
+import cv2
+import os
+from sys import platform
+import argparse
+import time
+
+# Import Openpose (Windows/Ubuntu/OSX)
+dir_path = os.path.dirname(os.path.realpath(__file__))
+try:
+    # Windows Import
+    if platform == "win32":
+        # Change these variables to point to the correct folder (Release/x64 etc.)
+        sys.path.append(dir_path + '/../../python/openpose/Release');
+        os.environ['PATH']  = os.environ['PATH'] + ';' + dir_path + '/../../x64/Release;' +  dir_path + '/../../bin;'
+        import pyopenpose as op
+    else:
+        # Change these variables to point to the correct folder (Release/x64 etc.)
+        sys.path.append('../../python');
+        # If you run `make install` (default path is `/usr/local/python` for Ubuntu), you can also access the OpenPose/python module from there. This will install OpenPose and the python library at your desired installation path. Ensure that this is in your python path in order to use it.
+        # sys.path.append('/usr/local/python')
+        from openpose import pyopenpose as op
+except ImportError as e:
+    print('Error: OpenPose library could not be found. Did you enable `BUILD_PYTHON` in CMake and have this Python script in the right folder?')
+    raise e
+
+# Flags
+parser = argparse.ArgumentParser()
+parser.add_argument("--image_dir", default="../../../examples/media/", help="Process a directory of images. Read all standard formats (jpg, png, bmp, etc.).")
+parser.add_argument("--no_display", default=False, help="Enable to disable the visual display.")
+args = parser.parse_known_args()
+
+# Custom Params (refer to include/openpose/flags.hpp for more parameters)
+params = dict()
+params["model_folder"] = "../../../models/"
+
+# Add others in path?
+for i in range(0, len(args[1])):
+    curr_item = args[1][i]
+    if i != len(args[1])-1: next_item = args[1][i+1]
+    else: next_item = "1"
+    if "--" in curr_item and "--" in next_item:
+        key = curr_item.replace('-','')
+        if key not in params:  params[key] = "1"
+    elif "--" in curr_item and "--" not in next_item:
+        key = curr_item.replace('-','')
+        if key not in params: params[key] = next_item
+
+# Construct it from system arguments
+# op.init_argv(args[1])
+# oppython = op.OpenposePython()
+
+# Starting OpenPose
+opWrapper = op.WrapperPython()
+opWrapper.configure(params)
+opWrapper.start()
+
+# Read frames on directory
+imagePaths = op.get_images_on_directory(args[0].image_dir);
+start = time.time()
+
+# Process and display images
+for imagePath in imagePaths:
+    datum = op.Datum()
+    imageToProcess = cv2.imread(imagePath)
+    datum.cvInputData = imageToProcess
+    opWrapper.emplaceAndPop([datum])
+
+    print("Body keypoints: \n" + str(datum.poseKeypoints))
+
+    if not args[0].no_display:
+        cv2.imshow("OpenPose 1.4.0 - Tutorial Python API", datum.cvOutputData)
+        key = cv2.waitKey(15)
+        if key == 27: break
+
+end = time.time()
+print("OpenPose demo successfully finished. Total time: " + str(end - start) + " seconds")

examples/tutorial_api_python/5_keypoints_from_images_multi_gpu.py

@@ -0,0 +1,103 @@
+# From Python
+# It requires OpenCV installed for Python
+import sys
+import cv2
+import os
+from sys import platform
+import argparse
+import time
+
+# Import Openpose (Windows/Ubuntu/OSX)
+dir_path = os.path.dirname(os.path.realpath(__file__))
+try:
+    # Windows Import
+    if platform == "win32":
+        # Change these variables to point to the correct folder (Release/x64 etc.)
+        sys.path.append(dir_path + '/../../python/openpose/Release');
+        os.environ['PATH']  = os.environ['PATH'] + ';' + dir_path + '/../../x64/Release;' +  dir_path + '/../../bin;'
+        import pyopenpose as op
+    else:
+        # Change these variables to point to the correct folder (Release/x64 etc.)
+        sys.path.append('../../python');
+        # If you run `make install` (default path is `/usr/local/python` for Ubuntu), you can also access the OpenPose/python module from there. This will install OpenPose and the python library at your desired installation path. Ensure that this is in your python path in order to use it.
+        # sys.path.append('/usr/local/python')
+        from openpose import pyopenpose as op
+except ImportError as e:
+    print('Error: OpenPose library could not be found. Did you enable `BUILD_PYTHON` in CMake and have this Python script in the right folder?')
+    raise e
+
+# Flags
+parser = argparse.ArgumentParser()
+parser.add_argument("--image_dir", default="../../../examples/media/", help="Process a directory of images. Read all standard formats (jpg, png, bmp, etc.).")
+parser.add_argument("--no_display", default=False, help="Enable to disable the visual display.")
+args = parser.parse_known_args()
+
+# Custom Params (refer to include/openpose/flags.hpp for more parameters)
+params = dict()
+params["model_folder"] = "../../../models/"
+
+# Add others in path?
+for i in range(0, len(args[1])):
+    curr_item = args[1][i]
+    if i != len(args[1])-1: next_item = args[1][i+1]
+    else: next_item = "1"
+    if "--" in curr_item and "--" in next_item:
+        key = curr_item.replace('-','')
+        if key not in params:  params[key] = "1"
+    elif "--" in curr_item and "--" not in next_item:
+        key = curr_item.replace('-','')
+        if key not in params: params[key] = next_item
+
+# Construct it from system arguments
+# op.init_argv(args[1])
+# oppython = op.OpenposePython()
+
+# Starting OpenPose
+opWrapper = op.WrapperPython()
+opWrapper.configure(params)
+opWrapper.start()
+
+# Read frames on directory
+imagePaths = op.get_images_on_directory(args[0].image_dir);
+
+# Read number of GPUs in your system
+numberGPUs = op.get_gpu_number()
+start = time.time()
+
+# Process and display images
+for imageBaseId in range(0, len(imagePaths), numberGPUs):
+
+    # Create datums
+    datums = []
+
+    # Read and push images into OpenPose wrapper
+    for gpuId in range(0, numberGPUs):
+
+        imageId = imageBaseId+gpuId
+        if imageId < len(imagePaths):
+
+            imagePath = imagePaths[imageBaseId+gpuId]
+            datum = op.Datum()
+            imageToProcess = cv2.imread(imagePath)
+            datum.cvInputData = imageToProcess
+            datums.append(datum)
+            opWrapper.waitAndEmplace([datums[-1]])
+
+    # Retrieve processed results from OpenPose wrapper
+    for gpuId in range(0, numberGPUs):
+
+        imageId = imageBaseId+gpuId
+        if imageId < len(imagePaths):
+
+            datum = datums[gpuId]
+            opWrapper.waitAndPop([datum])
+
+            print("Body keypoints: \n" + str(datum.poseKeypoints))
+
+            if not args[0].no_display:
+                cv2.imshow("OpenPose 1.4.0 - Tutorial Python API", datum.cvOutputData)
+                key = cv2.waitKey(15)
+                if key == 27: break
+
+end = time.time()
+print("OpenPose demo successfully finished. Total time: " + str(end - start) + " seconds")