Improve scripts, and add python 2 & 3 support to execution

[edwardjrp]

No description provided.

[robb-romans]

Thanks for the contribution! Am reviewing.

[robb-romans]

I've tested this in Python 2 and 3. Seems to work in v3. In Python 2.7.14 I get the following error:

$ ./script/add-assets ~/Deconst/nexus-control
Python 2.7.14
ENDPOINT | http://localhost:9000
PROD_CONTENT_URL | http://integrated_content_1:8080/
Python 2.7.14
PYTHON VERSION |
/Deconst/integrated/script/include/setup.sh: line 45: [: : integer expression expected
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 291, in load
    **kw)
  File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Unable to issue an API key.

May I suggest making something like the following change?

1 file changed, 1 insertion(+), 1 deletion(-)
script/include/setup.sh | 2 +-

modified   script/include/setup.sh
@@ -21,7 +21,7 @@ fi
 
 export PROD_CONTENT_URL=http://${PROD_CONTENT_NAME}:8080/
 export STAGING_CONTENT_URL=http://${STAGING_CONTENT_NAME}:8080/
-export PYVER=$(python --version | awk '{print $2}' | cut -c1)
+export PYVER=$(python -c 'import sys; print(sys.version_info[0])')
 
 apikey() {
   local KEYNAME="${1:-}"

[edwardjrp]

Will do thanks @robb-romans

[edwardjrp]

@robb-romans Change made please review

[robb-romans]

Thanks @edwardjrp! PYVER is now working for me in v2 and v3.

I cloned https://github.com/rackerlabs/docs-dedicated-networking and added a _deconst.json file to the dir. When running ./script/add-raml /Users/robb/Rcbops/upstream/docs-dedicated-networking I get:

ENDPOINT | http://localhost:9000
PROD_CONTENT_URL | http://integrated_content_1:8080/
Python 2.7.14
PYTHON VERSION |
CONTENT_ID_BASE | https://github.com/rackerlabs/docs-dedicated-networking
CONTENT_ROOT | /Users/robb/Rcbops/upstream/docs-dedicated-networking
CONTAINER_ROOT | /Users/robb/Rcbops/upstream/docs-dedicated-networking
CONTENT_URL | http://integrated_content_1:8080/
COMPOSE_NETWORK_NAME | integrated_default
"docker run" requires at least 1 argument.
See 'docker run --help'.

Usage:  docker run [OPTIONS] IMAGE [COMMAND] [ARG...] [flags]

Run a command in a new container

I ma be missing a step or not testing correctly, though. Should it be able to find your docker.io/edwardjrp/raml-preparer image?

[edwardjrp]

@robb-romans i had that image as private in my repo, its public now. Is there another repo to host this that is the prefered one?

[robb-romans]

As written this commented line breaks the command (line does not continue). Please remove or move to the end of the statement.

[robb-romans]

OK, thanks! I got further after pulling your image. See comment inline about removing a line.

The source I tested is RAML 0.8, so it failed. NB: raml2html 3.x supports 0.8.
Questions: do you intend to support 0.8, and, would you be willing to add some logic to fail gracefully and continue?

_sourceToRamlObj: only RAML 1.0 is supported!
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/preparer/ramlpreparer/__main__.py", line 8, in <module>
    main()
  File "/preparer/ramlpreparer/__init__.py", line 58, in main
    each_envelope = enveloper(path_name, base_location)
  File "/preparer/ramlpreparer/deconstraml.py", line 32, in enveloper
    the_envelope = envelope_writer.parsing_html(the_html)
  File "/preparer/ramlpreparer/builders/envelope_writer.py", line 235, in parsing_html
    that_page = tocbuilder.parse_it(contents2)
  File "/preparer/ramlpreparer/builders/tocbuilder.py", line 108, in parse_it
    cleared_tags = body_tag.findAll(regex101)
AttributeError: 'NoneType' object has no attribute 'findAll'

[robb-romans]

I'm not an expert on RAML best practices. Looks like 1.0 was announced (https://raml.org/blogs/raml-10-here) in late 2015. Perhaps we could convert all our existing 0.8 docs to 1.0. @nimbinatus thoughts?

[nimbinatus]

Yes, supporting 1.0 moving forward would be the ideal state. The RAML preparer as it stands now was only supposed to support 1.0+ after discussion with some different teams; that decision may have to be revisited as needed when new projects adopt the platform.

I will be doing PR reviews here and on the RAML preparer today; sorry for the delay in getting to them. Images for deconst are currently hosted at quay.io; the preparer will migrate there when the code is in a state to release.

[nimbinatus]

Some small wording nits, but this matches what I've been testing with in terms of code :) Thanks!

[nimbinatus]

Once we're all settled with the RAML preparer, we'll want to use the quay.io/deconst namespace to host the image. This works for now, though :)

[nimbinatus]

Nit: s/Jekyll/RAML

[edwardjrp]

Updated

[nimbinatus]

This path is to the content repository hosting RAML files, actually. So:

path - Path to a RAML content repository

[edwardjrp]

Updated

[nimbinatus]

Re: the findAll traceback: I'm running into the same problem, but I'm thinking it's a code issue at first glance, though it could be a package change. Looking at it in deconst/deconst-raml-preparer#1

[edwardjrp]

@robb-romans in the error mentioned above i dont see that the repository you are passing has raml file in it.

As to support latest version i just added a ticket to keep track and make updates on a separate PR.
https://jira.rax.io/browse/DE-26

[nimbinatus]

I think this is all correct, but we're currently blocked by deconst/deconst-raml-preparer#1, which is blocked by deconst/deconst-raml-preparer#2. Working on getting those merged today so this one can be merged.

[nimbinatus]

Approving this; it looks like the error has more to do with the code in the preparer than this change.