0.20.0 release

--show now always lists programs installed in given location, possibly
after installing some.

.travis.yml

@@ -12,11 +12,11 @@ matrix:
       language: generic
 
 install:
-  - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then sudo pip2 install pyflakes pep8 coverage coveralls nose; fi
+  - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then sudo pip2 install pyflakes pycodestyle coverage coveralls nose --ignore-installed; fi
-  - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then pip install pyflakes pep8 coverage coveralls nose; fi
+  - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then pip install pyflakes pycodestyle coverage coveralls nose; fi
 
 script:
   - pyflakes .
-  - pep8 .
+  - pycodestyle .
   - nosetests
   - coveralls

CHANGELOG.md

@@ -0,0 +1,243 @@
+# Changelog
+
+## 0.20.0 (2018-09-10)
+
+### New features and improvements
+
+* Added support for LuaRocks 3.0.0 - 3.0.2. `latest` and `^` aliases still
+  point to `2.4.4`.
+* `--show` can now be used when installing programs; all programs present
+  in the given location are listed after installation is complete.
+
+### Fixes
+
+* `package.path` for Lua 5.3+ built with hererocks now includes `./?/init.lua`,
+  just as with standard Lua.
+
+## 0.19.0 (2018-07-11)
+
+### New features and improvements
+
+* Added support for Lua 5.4.0-work1 and 5.4.0-work2.
+* Added support for Lua 5.3.5.
+* Added patches for Lua 5.3.4:
+
+    - `Lua crashes when building sequences with more than 2^30 elements`
+    - `Table length computation overflows for sequences larger than 2^31
+      elements`
+    - `Memory-allocation error when resizing a table can leave it in an
+      inconsistent state`
+
+## 0.18.0 (2018-03-13)
+
+### New features and improvements
+
+* Added support for LuaRocks 2.4.4.
+* Using `3` as version when installing LuaRocks now uses master branch of the
+  LuaRocks git repo instead of `luarocks-3`.
+* Added `lua_pushcclosure should not call the garbage collector when n is zero`
+  patch for Lua 5.3.4.
+
+## 0.17.0 (2017-09-13)
+
+### New features and improvements
+
+* Added support for LuaRocks 2.4.3.
+* When installing PUC-Rio Lua from a git repo or local sources, source files
+  are expected to be in root directory instead of `src`. This allows one to
+  install Lua from the default Lua repo at github.com/lua/lua.
+
+## 0.16.0 (2017-06-10)
+
+### New features and improvements
+
+* Added support for LuaJIT 2.0.5 and 2.1.0-beta3.
+* Added fallback to a mirror for PUC-Rio Lua download.
+* Added `--timeout` option for downloads.
+
+## 0.15.0 (2017-04-02)
+
+### New features and improvements
+
+* When building Lua with custom cflags, hererocks now instructs LuaRocks to use
+  same flags when building C modules.
+
+## 0.14.0 (2017-01-31)
+
+### New features and improvements
+
+* Added support for Lua 5.3.4.
+
+## 0.13.1 (2017-01-09)
+
+### Fixes
+
+* Fixed an error when setting up cl.exe and hererocks has been installed using
+  a version of pip that passes `prefix/hererocks` instead of
+  `prefix/hererocks-script.py` as `sys.argv[0]`.
+
+## 0.13.0 (2016-12-22)
+
+### New features and improvements
+
+* Added support for LuaRocks 2.4.2.
+* LuaRocks is now configured to allow using config in user home directory.
+  In particular, API keys for `luarocks upload` command are now properly cached.
+
+## 0.12.0 (2016-11-04)
+
+### New features and improvements
+
+* Added support for LuaRocks 2.4.1.
+* Updated URL for LuaRocks downloads and default git repo.
+
+## 0.11.0 (2016-09-10)
+
+### New features and improvements
+
+* Added support for LuaRocks 2.4.0.
+* Added two new patches for bugs in Lua 5.3.3, try `--patch`.
+* Caching of downloads now works on Unix-like systems even when `$HOME` is
+  unset (#28).
+
+## 0.10.0 (2016-07-14)
+
+### New features and improvements
+
+* hererocks now creates activation scripts a-la virtualenv in `<location>/bin`.
+  Bash, Zsh, Dash, Fish, Csh, Batch, and PowerShell are supported.
+* Lua 5.3.3 can now be patched to fix a bug (`Expression list with four or more
+  expressions in a 'for' loop can crash the interpreter`), try `--patch`.
+
+## 0.9.0 (2016-06-21)
+
+### New features and improvements
+
+* Added support for LuaRocks 2.0.13.
+
+### Fixes
+
+* Fixed occasional SHA256 mismatches when downloading LuaJIT (#27).
+
+## 0.8.1 (2016-06-12)
+
+### Fixes
+
+* Fixed error when installing from non-default git repo on Windows.
+
+## 0.8.0 (2016-06-07)
+
+### New features and improvements
+
+* Added support for Lua 5.3.3.
+
+## 0.7.0 (2016-05-03)
+
+### New features and improvements
+
+* Windows support with automatic Visual Studio setup.
+* New values for `--target` option for selecting Visual Studio version and
+  target architecture.
+* New `--show` option for listing programs installed in a location.
+* New `--patch` option for applying official patches for bugs in Lua (#21).
+* Documented `--downloads` and `--builds` options, may be useful for caching.
+
+### Fixes
+
+* Fixed an error when a command failed when using Python 3 (#15).
+* Fixed error when running with `HOME` environment variable undefined (#24).
+
+## 0.6.2 (2016-03-22)
+
+### Fixes
+
+Fixed a bug that resulted in Lua being built without compatibility flags (#14).
+
+## 0.6.1 (2016-03-22) [yanked]
+
+## 0.6.0 (2016-03-20)
+
+### Breaking changes
+
+* `hererocks --luajit 2.1` now installs LuaJIT 2.1.0-beta2 instead of
+  using v2.1 git branch. Use `hererocks --luajit @v2.1` to get old behaviour.
+
+### New features and improvements
+
+* LuaJIT versions 2.1.0-beta1 - 2.1.0-beta2 are now supported.
+* Lua is now built manually (`make` is not run).
+* OS X 10.4+ support for Lua 5.1.0 - 5.1.2.
+* SHA256 checksums for downloaded archives are now verified (#13).
+* `--no-readline` flag for building Lua without readline library.
+* Lua archives are now downloaded using HTTPS.
+
+### Fixes
+
+* `luajit.h` is installed for LuaJIT (#11, #12).
+* `jit.*` modules work correctly for LuaJIT 2.0.1.
+
+## 0.5.0 (2016-01-11)
+
+### New features and improvements
+
+LuaRocks 2.3.0 is now supported.
+
+## 0.4.0 (2016-01-03)
+
+### New features and improvements
+
+* Documented --no-git-cache option.
+
+### Fixes
+
+* Fixed error when installing Lua 5.1 on OS X, thanks to @xpol.
+
+## 0.3.1 (2015-12-22)
+
+### Fixes
+
+* Fetch LuaJIT from GitHub mirror archive to avoid 'Connection refused' errors
+  when installing it on Travis.
+
+## 0.3.0 (2015-12-19)
+
+### New features and improvements
+
+* LuaRocks versions 2.0.8 - 2.0.12 are now supported.
+* `--compat=none` now turns off Lua 5.0 compatibility options when installing
+  Lua 5.1
+* Default git repos are cached.
+* New `--cflags` option for adding custom compiler flags when compiling Lua and
+  LuaJIT.
+
+### Fixes
+
+* LuaJIT `jit.*` modules are now properly installed.
+
+## 0.2.0 (2015-12-02)
+
+### New features and improvements
+
+* Lua 5.3.2 is now supported.
+* Versions of installed programs are showed in status messages.
+
+## 0.1.0 (2015-11-29)
+
+### Breaking changes
+
+* Removed `-c` and `-t` shortcuts.
+
+### New features and improvements
+
+* `--verbose` flag that prints commands hererocks runs.
+* hererocks now checks if requested versions are already installed, and skips
+  installation in that case. Override using `--ignore-installed/-i` flag.
+* PUC Rio Lua now has default git URI.
+
+### Fixes
+
+* Installing from a git branch now works with newer git versions.
+
+## 0.0.3 (2015-08-14)
+
+The first release.

README.rst

@@ -95,7 +95,7 @@ Lua and LuaJIT have some flags that add compatibility with other Lua versions. L
 Installing standard PUC-Rio Lua
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Available versions: 5.1 - 5.1.5, 5.2.0 - 5.2.4, 5.3.0 - 5.3.4.
+Available versions: 5.1 - 5.1.5, 5.2.0 - 5.2.4, 5.3.0 - 5.3.5, 5.4.0-work1 - 5.4.0-work2 (5.4 and 5.4.0 are aliases for 5.4.0-work2). ``latest`` and ``^`` version aliases point to ``5.3.5``.
 
 Use ``5.1.0`` to install Lua ``5.1`` which was released without patch version for some reason.
 
@@ -104,12 +104,12 @@ When building Lua, ``hererocks`` tries to emulate a sensible ``make`` target. Th
 Installing LuaJIT
 ^^^^^^^^^^^^^^^^^
 
-Available versions: 2.0.0 - 2.0.5, 2.1.0-beta1 - 2.1.0-beta3.
+Available versions: 2.0.0 - 2.0.5, 2.1.0-beta1 - 2.1.0-beta3. ``latest`` and ``^`` version aliases point to ``2.0.5``.
 
 Installing LuaRocks
 ^^^^^^^^^^^^^^^^^^^
 
-Available versions: 2.0.8 - 2.0.13, 2.1.0 - 2.1.2, 2.2.0 - 2.2.2, 2.3.0, 2.4.0 - 2.4.4, 3 (installs from ``master`` branch of the LuaRocks git repository).
+Available versions: 2.0.8 - 2.0.13, 2.1.0 - 2.1.2, 2.2.0 - 2.2.2, 2.3.0, 2.4.0 - 2.4.4, 3.0.0 - 3.0.2. ``latest`` and ``^`` version aliases point to ``2.4.4``.
 
 Version 2.0.8 does not support Lua 5.2. Versions 2.0.8 - 2.1.2 do not support Lua 5.3.
 

appveyor.yml

@@ -7,11 +7,11 @@ environment:
 
 build_script:
   - PATH %CD%\here\bin;C:\mingw\bin;C:\python%Python%;C:\python%Python%\scripts;%PATH%
-  - pip install pyflakes pep8 coverage coveralls nose
+  - pip install pyflakes pycodestyle coverage coveralls nose
 
 test_script:
   - pyflakes .
-  - pep8 .
+  - pycodestyle .
   - nosetests
   - coverage report
   - if not "%COVERALLS_REPO_TOKEN%"=="" coveralls

hererocks.py

@@ -35,7 +35,7 @@ if os.name == "nt":
     except ImportError:
         import winreg
 
-hererocks_version = "Hererocks 0.18.0"
+hererocks_version = "Hererocks 0.20.0"
 __all__ = ["main"]
 
 opts = None
@@ -154,7 +154,7 @@ def write_activation_scripts():
         "LOCATION_DQ": opts.location.replace("\\", "\\\\").replace('"', '\\"'),
         "LOCATION_SQ": opts.location.replace("'", "'\\''"),
         "LOCATION_NESTED_SQ": opts.location.replace("'", "'\\''").replace("'", "'\\''"),
-        "LOCATION_PAREN": re.sub("[&,=()]", "^\g<0>", opts.location)
+        "LOCATION_PAREN": re.sub("[&,=()]", r"^\g<0>", opts.location)
     }
 
     for template_name in template_names:
@@ -165,9 +165,7 @@ def write_activation_scripts():
             script_handle.write(script)
 
 def is_executable(path):
-    return (os.path.exists(path) and
+    return os.path.exists(path) and os.access(path, os.F_OK | os.X_OK) and not os.path.isdir(path)
-            os.access(path, os.F_OK | os.X_OK) and
-            not os.path.isdir(path))
 
 def program_exists(prog):
     path = os.environ.get("PATH", os.defpath)
@@ -198,7 +196,6 @@ platform_to_lua_target = {
 def using_cl():
     return opts.target.startswith("vs")
 
-
 def get_default_lua_target():
     for plat, lua_target in platform_to_lua_target.items():
         if sys.platform.startswith(plat):
@@ -421,6 +418,14 @@ def sha256_of_file(filename):
 
     return hashlib.sha256(contents).hexdigest()
 
+def strip_extensions(filename):
+    if filename.endswith(".zip"):
+        return filename[:-len(".zip")]
+    elif filename.endswith(".tar.gz"):
+        return filename[:-len(".tar.gz")]
+    else:
+        return filename
+
 class Program(object):
     def __init__(self, version):
         version = self.translations.get(version, version)
@@ -430,7 +435,6 @@ class Program(object):
             self.source = "release"
             self.fetched = False
             self.version = version
-            self.fixed_version = version
             self.version_suffix = " " + version
         elif "@" in version:
             # Version from a git repo.
@@ -499,15 +503,6 @@ class Program(object):
         if need_checkout and ref != "master":
             run("git", "checkout", ref)
 
-    def get_download_name(self):
-        return self.name + "-" + self.fixed_version + ("-win32" if self.win32_zip else "")
-
-    def get_file_name(self):
-        return self.get_download_name() + (".zip" if self.win32_zip else ".tar.gz")
-
-    def get_download_url(self, base_url):
-        return base_url + "/" + self.get_file_name()
-
     def fetch(self):
         if self.fetched:
             return
@@ -520,18 +515,17 @@ class Program(object):
             return
 
         if opts.downloads is None:
-            archive_name = os.path.join(temp_dir, self.get_file_name())
+            archive_name = os.path.join(temp_dir, self.get_download_name())
         else:
             if not os.path.exists(opts.downloads):
                 os.makedirs(opts.downloads)
 
-            archive_name = os.path.join(opts.downloads, self.get_file_name())
+            archive_name = os.path.join(opts.downloads, self.get_download_name())
 
         if opts.downloads and os.path.exists(archive_name):
             print("Fetching {}{} (cached)".format(self.title, self.version_suffix))
         else:
-            for base_url in self.downloads:
+            for url in self.get_download_urls():
-                url = self.get_download_url(base_url)
                 print("Fetching {}{} from {}".format(self.title, self.version_suffix, url))
 
                 try:
@@ -544,7 +538,7 @@ class Program(object):
                 sys.exit(1)
 
         print("Verifying SHA256 checksum")
-        expected_checksum = self.checksums[self.get_file_name()]
+        expected_checksum = self.checksums[self.get_download_name()]
         observed_checksum = sha256_of_file(archive_name)
         if expected_checksum != observed_checksum:
             message = "SHA256 checksum mismatch for {}\nExpected: {}\nObserved: {}".format(
@@ -555,14 +549,14 @@ class Program(object):
             else:
                 sys.exit("Error: " + message)
 
-        if self.win32_zip:
+        if archive_name.endswith(".zip"):
             archive = zipfile.ZipFile(archive_name)
         else:
             archive = tarfile.open(archive_name, "r:gz")
 
         archive.extractall(temp_dir)
         archive.close()
-        os.chdir(os.path.join(temp_dir, self.get_download_name()))
+        os.chdir(os.path.join(temp_dir, strip_extensions(self.get_download_name())))
         self.fetched = True
 
     def set_identifiers(self):
@@ -697,6 +691,10 @@ class Lua(Program):
             os.path.join(module_path, "?", "init.lua")
         ]
         module_path_parts.insert(0 if local_paths_first else 2, os.path.join(".", "?.lua"))
+
+        if self.major_version in ["5.3", "5.4"]:
+            module_path_parts.append(os.path.join(".", "?", "init.lua"))
+
         self.package_path = ";".join(module_path_parts)
 
         cmodule_path = os.path.join(opts.location, "lib", "lua", self.major_version)
@@ -877,22 +875,25 @@ class Patch(object):
 class RioLua(Lua):
     name = "lua"
     title = "Lua"
-    downloads = ["http://www.lua.org/ftp", "http://webserver2.tecgraf.puc-rio.br/lua/mirror/ftp"]
+    base_download_urls = ["http://www.lua.org/ftp", "http://webserver2.tecgraf.puc-rio.br/lua/mirror/ftp"]
-    win32_zip = False
+    work_base_download_url = "http://www.lua.org/work"
     default_repo = "https://github.com/lua/lua"
     versions = [
         "5.1", "5.1.1", "5.1.2", "5.1.3", "5.1.4", "5.1.5",
         "5.2.0", "5.2.1", "5.2.2", "5.2.3", "5.2.4",
-        "5.3.0", "5.3.1", "5.3.2", "5.3.3", "5.3.4"
+        "5.3.0", "5.3.1", "5.3.2", "5.3.3", "5.3.4", "5.3.5",
+        "5.4.0", "5.4.0-work1", "5.4.0-work2"
     ]
     translations = {
-        "5": "5.3.4",
+        "5": "5.3.5",
         "5.1": "5.1.5",
         "5.1.0": "5.1",
         "5.2": "5.2.4",
-        "5.3": "5.3.4",
+        "5.3": "5.3.5",
-        "^": "5.3.4",
+        "5.4": "5.4.0-work2",
-        "latest": "5.3.4"
+        "5.4.0": "5.4.0-work2",
+        "^": "5.3.5",
+        "latest": "5.3.5"
     }
     checksums = {
         "lua-5.1.tar.gz"        : "7f5bb9061eb3b9ba1e406a5aa68001a66cb82bac95748839dc02dd10048472c1",
@@ -911,6 +912,9 @@ class RioLua(Lua):
         "lua-5.3.2.tar.gz"      : "c740c7bb23a936944e1cc63b7c3c5351a8976d7867c5252c8854f7b2af9da68f",
         "lua-5.3.3.tar.gz"      : "5113c06884f7de453ce57702abaac1d618307f33f6789fa870e87a59d772aca2",
         "lua-5.3.4.tar.gz"      : "f681aa518233bc407e23acf0f5887c884f17436f000d453b2491a9f11a52400c",
+        "lua-5.3.5.tar.gz"      : "0c2eed3f960446e1a3e4b9a1ca2f3ff893b6ce41942cf54d5dd59ab4b3b058ac",
+        "lua-5.4.0-work1.tar.gz": "ada03980481110bfde44b3bd44bde4b03d72c84318b34d657b5b5a91ddb3912c",
+        "lua-5.4.0-work2.tar.gz": "68b7e8f1ff561b9a7e1c29de26ff99ac2a704773c0965a4fe1800b7657d5a057",
     }
     all_patches = {
         "When loading a file, Lua may call the reader function again after it returned end of input": """
@@ -1102,6 +1106,91 @@ class RioLua(Lua):
             -  luaC_checkGC(L);
                lua_unlock(L);
              }
+        """,
+        "Lua crashes when building sequences with more than 2^30 elements": """
+            ltable.c:
+            @@ -223,7 +223,9 @@
+               unsigned int na = 0;  /* number of elements to go to array part */
+               unsigned int optimal = 0;  /* optimal size for array part */
+               /* loop while keys can fill more than half of total size */
+            -  for (i = 0, twotoi = 1; *pna > twotoi / 2; i++, twotoi *= 2) {
+            +  for (i = 0, twotoi = 1;
+            +       twotoi > 0 && *pna > twotoi / 2;
+            +       i++, twotoi *= 2) {
+                 if (nums[i] > 0) {
+                   a += nums[i];
+                   if (a > twotoi/2) {  /* more than half elements present? */
+        """,
+        "Table length computation overflows for sequences larger than 2^31 elements": """
+            ltable.h:
+            @@ -56,3 +56,3 @@
+             LUAI_FUNC int luaH_next (lua_State *L, Table *t, StkId key);
+            -LUAI_FUNC int luaH_getn (Table *t);
+            +LUAI_FUNC lua_Unsigned luaH_getn (Table *t);
+
+            ltable.c:
+            @@ -614,4 +614,4 @@
+
+            -static int unbound_search (Table *t, unsigned int j) {
+            -  unsigned int i = j;  /* i is zero or a present index */
+            +static lua_Unsigned unbound_search (Table *t, lua_Unsigned j) {
+            +  lua_Unsigned i = j;  /* i is zero or a present index */
+               j++;
+            @@ -620,3 +620,3 @@
+                 i = j;
+            -    if (j > cast(unsigned int, MAX_INT)/2) {  /* overflow? */
+            +    if (j > l_castS2U(LUA_MAXINTEGER) / 2) {  /* overflow? */
+                   /* table was built with bad purposes: resort to linear search */
+            @@ -630,3 +630,3 @@
+               while (j - i > 1) {
+            -    unsigned int m = (i+j)/2;
+            +    lua_Unsigned m = (i+j)/2;
+                 if (ttisnil(luaH_getint(t, m))) j = m;
+            @@ -642,3 +642,3 @@
+             */
+            -int luaH_getn (Table *t) {
+            +lua_Unsigned luaH_getn (Table *t) {
+               unsigned int j = t->sizearray;
+        """,
+        "Memory-allocation error when resizing a table can leave it in an inconsistent state":
+        """
+            ltable.c:
+            @@ -332,17 +332,34 @@
+             }
+
+
+            +typedef struct {
+            +  Table *t;
+            +  unsigned int nhsize;
+            +} AuxsetnodeT;
+            +
+            +
+            +static void auxsetnode (lua_State *L, void *ud) {
+            +  AuxsetnodeT *asn = cast(AuxsetnodeT *, ud);
+            +  setnodevector(L, asn->t, asn->nhsize);
+            +}
+            +
+            +
+             void luaH_resize (lua_State *L, Table *t, unsigned int nasize,
+                                                       unsigned int nhsize) {
+               unsigned int i;
+               int j;
+            +  AuxsetnodeT asn;
+               unsigned int oldasize = t->sizearray;
+               int oldhsize = allocsizenode(t);
+               Node *nold = t->node;  /* save old hash ... */
+               if (nasize > oldasize)  /* array part must grow? */
+                 setarrayvector(L, t, nasize);
+               /* create new hash part with appropriate size */
+            -  setnodevector(L, t, nhsize);
+            +  asn.t = t; asn.nhsize = nhsize;
+            +  if (luaD_rawrunprotected(L, auxsetnode, &asn) != LUA_OK) {  /* mem. error? */
+            +    setarrayvector(L, t, oldasize);  /* array back to its original size */
+            +    luaD_throw(L, LUA_ERRMEM);  /* rethrow memory error */
+            +  }
+               if (nasize < oldasize) {  /* array part must shrink? */
+                 t->sizearray = nasize;
+                 /* re-insert elements from vanishing slice */
         """
     }
     patches_per_version = {
@@ -1123,9 +1212,12 @@ class RioLua(Lua):
             ],
             "4": [
                 "Wrong code generated for a 'goto' followed by a label inside an 'if'",
+                "Lua crashes when building sequences with more than 2^30 elements",
+                "Table length computation overflows for sequences larger than 2^31 elements",
                 "Lua does not check GC when creating error messages",
                 "Dead keys with nil values can stay in weak tables",
-                "lua_pushcclosure should not call the garbage collector when n is zero"
+                "lua_pushcclosure should not call the garbage collector when n is zero",
+                "Memory-allocation error when resizing a table can leave it in an inconsistent state"
             ]
         }
     }
@@ -1146,6 +1238,15 @@ class RioLua(Lua):
         else:
             self.dll_file = None
 
+    def get_download_name(self):
+        return "{}-{}.tar.gz".format(self.name, self.version)
+
+    def get_download_urls(self):
+        if self.version.startswith("5.4.0-work"):
+            return ["{}/{}".format(self.work_base_download_url, self.get_download_name())]
+        else:
+            return ["{}/{}".format(base_download_url, self.get_download_name()) for base_download_url in self.base_download_urls]
+
     def get_source_files_prefix(self):
         # When installing PUC-Rio Lua from a git repo or local sources,
         # use directory structure of its GitHub mirror, where
@@ -1170,8 +1271,10 @@ class RioLua(Lua):
             self.compat = "none" if opts.compat == "none" else "default"
         elif self.major_version == "5.2":
             self.compat = "none" if opts.compat in ["none", "5.2"] else "default"
-        else:
+        elif self.major_version == "5.3":
             self.compat = "default" if opts.compat in ["default", "5.2"] else opts.compat
+        else:
+            self.compat = "default" if opts.compat in ["default", "5.3"] else opts.compat
 
     def add_compat_cflags_and_redefines(self):
         if self.major_version == "5.1":
@@ -1184,12 +1287,15 @@ class RioLua(Lua):
         elif self.major_version == "5.2":
             if self.compat == "default":
                 self.compat_cflags.append("-DLUA_COMPAT_ALL")
-        else:
+        elif self.major_version == "5.3":
             if self.compat in ["5.1", "all"]:
                 self.compat_cflags.append("-DLUA_COMPAT_5_1")
 
             if self.compat in ["default", "5.2", "all"]:
                 self.compat_cflags.append("-DLUA_COMPAT_5_2")
+        else:
+            if self.compat in ["default", "5.3", "all"]:
+                self.compat_cflags.append("-DLUA_COMPAT_5_3")
 
     def apply_patch(self, patch_name):
         patch = self.all_patches[patch_name]
@@ -1203,7 +1309,7 @@ class RioLua(Lua):
         regexps = [
             # Lua 5.1.x, but not Lua 5.1(.0)
             r'^\s*#define\s+LUA_RELEASE\s+"Lua 5\.1\.(\d)"\s*$',
-            # Lua 5.2.x and 5.3.x
+            # Lua 5.2.x+
             r'^\s*#define LUA_VERSION_RELEASE\s+"(\d)"\s*$'
         ]
 
@@ -1249,7 +1355,7 @@ class RioLua(Lua):
             applied, "" if applied == 1 else "es", len(patches)))
 
     def make(self):
-        if self.major_version == "5.3":
+        if self.major_version == "5.3" or self.major_version == "5.4":
             cc = ["gcc", "-std=gnu99"]
         else:
             cc = "gcc"
@@ -1386,8 +1492,7 @@ class RioLua(Lua):
 class LuaJIT(Lua):
     name = "LuaJIT"
     title = "LuaJIT"
-    downloads = ["https://github.com/LuaJIT/LuaJIT/archive"]
+    base_download_url = "https://github.com/LuaJIT/LuaJIT/archive"
-    win32_zip = False
     default_repo = "https://github.com/LuaJIT/LuaJIT"
     versions = [
         "2.0.0", "2.0.1", "2.0.2", "2.0.3", "2.0.4", "2.0.5",
@@ -1412,15 +1517,12 @@ class LuaJIT(Lua):
         "LuaJIT-2.1.0-beta3.tar.gz": "409f7fe570d3c16558e594421c47bdd130238323c9d6fd6c83dedd2aaeb082a8",
     }
 
-    def __init__(self, version):
+    def get_download_name(self):
-        super(LuaJIT, self).__init__(version)
-
-        if self.source == "release" and self.version == "2.0.1":
         # v2.0.1 tag is broken, use v2.0.1-fixed.
-            self.fixed_version = "2.0.1-fixed"
+        return "{}-{}.tar.gz".format(self.name, "2.0.1-fixed" if self.version == "2.0.1" else self.version)
 
-    def get_download_url(self, base_url):
+    def get_download_urls(self):
-        return base_url + "/v" + self.fixed_version + ".tar.gz"
+        return ["{}/v{}.tar.gz".format(self.base_download_url, "2.0.1-fixed" if self.version == "2.0.1" else self.version)]
 
     @staticmethod
     def major_version_from_version():
@@ -1513,15 +1615,15 @@ class LuaJIT(Lua):
 class LuaRocks(Program):
     name = "luarocks"
     title = "LuaRocks"
-    downloads = ["http://luarocks.github.io/luarocks/releases"]
+    base_download_url = "http://luarocks.github.io/luarocks/releases"
-    win32_zip = os.name == "nt"
     default_repo = "https://github.com/luarocks/luarocks"
     versions = [
         "2.0.8", "2.0.9", "2.0.10", "2.0.11", "2.0.12", "2.0.13",
         "2.1.0", "2.1.1", "2.1.2",
         "2.2.0", "2.2.1", "2.2.2",
         "2.3.0",
-        "2.4.0", "2.4.1", "2.4.2", "2.4.3", "2.4.4"
+        "2.4.0", "2.4.1", "2.4.2", "2.4.3", "2.4.4",
+        "3.0.0", "3.0.1", "3.0.2"
     ]
     translations = {
         "2": "2.4.4",
@@ -1530,7 +1632,8 @@ class LuaRocks(Program):
         "2.2": "2.2.2",
         "2.3": "2.3.0",
         "2.4": "2.4.4",
-        "3": "@master",
+        "3": "3.0.2",
+        "3.0": "3.0.2",
         "^": "2.4.4",
         "latest": "2.4.4"
     }
@@ -1571,8 +1674,20 @@ class LuaRocks(Program):
         "luarocks-2.4.3-win32.zip" : "08821ec39e7c3ad20f5b3d3e118ba8f1f5a7db6e6ad22e11eb5e8a2bdc95cbfb",
         "luarocks-2.4.4.tar.gz"    : "3938df33de33752ff2c526e604410af3dceb4b7ff06a770bc4a240de80a1f934",
         "luarocks-2.4.4-win32.zip" : "763d2fbe301b5f941dd5ea4aea485fb35e75cbbdceca8cc2f18726b75f9895c1",
+        "luarocks-3.0.0.tar.gz"    : "a43fffb997100f11cccb529a3db5456ce8dab18171a5cb3645f948147b6f64a1",
+        "luarocks-3.0.0-win32.zip" : "f5c6070f49f78ef61a2e5d6de353b34ef691ad4a6b45e065d5c85701a4a3a981",
+        "luarocks-3.0.1.tar.gz"    : "b989c4b60d6c9edcd65169e5e42fcffbd39cdbebe6b138fa5aea45102f8d9ec0",
+        "luarocks-3.0.1-win32.zip" : "af54263b8f71406d79556c880f3e2674e6690934a69cefbbdfd18710f05eeeaf",
+        "luarocks-3.0.2.tar.gz"    : "3836267eff2f85fb552234e966602b1e649c58f81f47c7de3785e071c8127f5a",
+        "luarocks-3.0.2-win32.zip" : "c9e93d7198f9ae7add331675d3d84fa1b61feb851814ee2a89b9930bd651bfb9",
     }
 
+    def get_download_name(self):
+        return "{}-{}{}".format(self.name, self.version, "-win32.zip" if os.name == "nt" else ".tar.gz")
+
+    def get_download_urls(self):
+        return ["{}/{}".format(self.base_download_url, self.get_download_name())]
+
     def is_luarocks_2_0(self):
         if self.source == "release":
             return self.versions.index(self.version) < self.versions.index("2.1.0")
@@ -1864,6 +1979,71 @@ class UseActualArgsFileAction(argparse.Action):
 
         main(args_content.split("\r\n")[1:])
 
+def install_programs(vs_already_set_up):
+    global temp_dir
+    temp_dir = tempfile.mkdtemp()
+
+    if (opts.lua or opts.luajit) and os.name == "nt" and not vs_already_set_up and using_cl():
+        setup_vs(opts.target)
+
+    start_dir = os.getcwd()
+    opts.location = os.path.abspath(opts.location)
+
+    if opts.downloads is not None:
+        opts.downloads = os.path.abspath(opts.downloads)
+
+    if opts.builds is not None:
+        opts.builds = os.path.abspath(opts.builds)
+
+    identifiers = get_installed_identifiers()
+
+    if not os.path.exists(os.path.join(opts.location, "bin")):
+        os.makedirs(os.path.join(opts.location, "bin"))
+
+    write_activation_scripts()
+
+    if opts.lua:
+        if "LuaJIT" in identifiers:
+            del identifiers["LuaJIT"]
+
+        if RioLua(opts.lua).update_identifiers(identifiers):
+            save_installed_identifiers(identifiers)
+
+        os.chdir(start_dir)
+
+    if opts.luajit:
+        if "lua" in identifiers:
+            del identifiers["lua"]
+
+        if LuaJIT(opts.luajit).update_identifiers(identifiers):
+            save_installed_identifiers(identifiers)
+
+        os.chdir(start_dir)
+
+    if opts.luarocks:
+        if LuaRocks(opts.luarocks).update_identifiers(identifiers):
+            save_installed_identifiers(identifiers)
+
+        os.chdir(start_dir)
+
+    remove_dir(temp_dir)
+    print("Done.")
+
+def show_location():
+    if os.path.exists(opts.location):
+        all_identifiers = get_installed_identifiers()
+
+        if all_identifiers:
+            print("Programs installed in {}:".format(opts.location))
+
+            for program in [RioLua, LuaJIT, LuaRocks]:
+                if program.name in all_identifiers:
+                    show_identifiers(all_identifiers[program.name])
+        else:
+            print("No programs installed in {}.".format(opts.location))
+    else:
+        print("{} does not exist.".format(opts.location))
+
 def main(argv=None):
     parser = argparse.ArgumentParser(
         description=hererocks_version + ", a tool for installing Lua and/or LuaRocks locally.",
@@ -1877,8 +2057,8 @@ def main(argv=None):
     parser.add_argument(
         "-l", "--lua", help="Version of standard PUC-Rio Lua to install. "
         "Version can be specified as a version number, e.g. 5.2 or 5.3.1. "
-        "Versions 5.1.0 - 5.3.4 are supported, "
+        "Versions 5.1.0 - 5.3.5 and 5.4.0-work1 - 5.4.0-work2 are supported. "
-        "'^' or 'latest' can be used to install the latest stable version. "
+        "'latest' and '^' are aliases for 5.3.5. "
         "If the argument contains '@', sources will be downloaded "
         "from a git repo using URI before '@' and using part after '@' as git reference "
         "to checkout, 'master' by default. "
@@ -1890,17 +2070,19 @@ def main(argv=None):
         "-j", "--luajit", help="Version of LuaJIT to install. "
         "Version can be specified in the same way as for standard Lua. "
         "Versions 2.0.0 - 2.1.0-beta3 are supported. "
-        "When installing from the LuaJIT main git repo its URI can be left out, "
+        "'latest' and '^' are aliases for to 2.0.5. "
-        "so that '@458a40b' installs from a commit and '@' installs from the master branch.")
+        "Default git repo is https://github.com/luajit/luajit. ")
     parser.add_argument(
         "-r", "--luarocks", help="Version of LuaRocks to install. "
-        "As with Lua, a version number (in range 2.0.8 - 2.4.4), '^', git URI with reference or "
+        "Version can be specified in the same way as for standard Lua. "
-        "a local path can be used. '3' can be used as a version number and installs from "
+        "Versions 2.0.8 - 3.0.2 are supported. "
-        "the 'master' branch of the standard LuaRocks git repo. "
+        "'latest' and '^' are aliases for 2.4.4. "
-        "Note that Lua 5.2 is not supported in LuaRocks 2.0.8 "
+        "Default git repo is https://github.com/luarocks/luarocks. "
-        "and Lua 5.3 is supported only since LuaRocks 2.2.0.")
+        "Note that Lua 5.2 is not supported in LuaRocks 2.0.8, "
+        "Lua 5.3 is supported only since LuaRocks 2.2.0, and Lua 5.4 is supported only since "
+        "LuaRocks 3.0.0.")
     parser.add_argument("--show", default=False, action="store_true",
-                        help="Instead of installing show programs already present in <location>")
+                        help="Show programs installed in <location>, possibly after installing new ones.")
     parser.add_argument("-i", "--ignore-installed", default=False, action="store_true",
                         help="Install even if requested version is already present.")
     parser.add_argument(
@@ -1962,79 +2144,17 @@ def main(argv=None):
     global opts
     opts = parser.parse_args(argv)
     if not opts.lua and not opts.luajit and not opts.luarocks and not opts.show:
-        parser.error("nothing to do")
+        parser.error("a version of Lua, LuaJIT, or LuaRocks needs to be specified unless --show is used")
 
     if opts.lua and opts.luajit:
         parser.error("can't install both PUC-Rio Lua and LuaJIT")
 
-    if (opts.lua or opts.luajit or opts.luarocks) and opts.show:
+    if opts.lua or opts.luajit or opts.luarocks:
-        parser.error("can't both install and show")
+        install_programs(argv is not None)
 
     if opts.show:
-        if os.path.exists(opts.location):
+        show_location()
-            all_identifiers = get_installed_identifiers()
 
-            if all_identifiers:
-                print("Programs installed in {}:".format(opts.location))
-
-                for program in [RioLua, LuaJIT, LuaRocks]:
-                    if program.name in all_identifiers:
-                        show_identifiers(all_identifiers[program.name])
-            else:
-                print("No programs installed in {}.".format(opts.location))
-        else:
-            print("Location does not exist.")
-
-        sys.exit(0)
-
-    global temp_dir
-    temp_dir = tempfile.mkdtemp()
-
-    if (opts.lua or opts.luajit) and os.name == "nt" and argv is None and using_cl():
-        setup_vs(opts.target)
-
-    start_dir = os.getcwd()
-    opts.location = os.path.abspath(opts.location)
-
-    if opts.downloads is not None:
-        opts.downloads = os.path.abspath(opts.downloads)
-
-    if opts.builds is not None:
-        opts.builds = os.path.abspath(opts.builds)
-
-    identifiers = get_installed_identifiers()
-
-    if not os.path.exists(os.path.join(opts.location, "bin")):
-        os.makedirs(os.path.join(opts.location, "bin"))
-
-    write_activation_scripts()
-
-    if opts.lua:
-        if "LuaJIT" in identifiers:
-            del identifiers["LuaJIT"]
-
-        if RioLua(opts.lua).update_identifiers(identifiers):
-            save_installed_identifiers(identifiers)
-
-        os.chdir(start_dir)
-
-    if opts.luajit:
-        if "lua" in identifiers:
-            del identifiers["lua"]
-
-        if LuaJIT(opts.luajit).update_identifiers(identifiers):
-            save_installed_identifiers(identifiers)
-
-        os.chdir(start_dir)
-
-    if opts.luarocks:
-        if LuaRocks(opts.luarocks).update_identifiers(identifiers):
-            save_installed_identifiers(identifiers)
-
-        os.chdir(start_dir)
-
-    remove_dir(temp_dir)
-    print("Done.")
     sys.exit(0)
 
 if __name__ == "__main__":

setup.cfg

@@ -1,2 +1,2 @@
-[pep8]
+[pycodestyle]
-ignore = E203,E302,E501
+ignore = E203,E302,E305,E501

setup.py

@@ -9,7 +9,7 @@ readme.close()
 
 setuptools.setup(
     name="hererocks",
-    version="0.18.0",
+    version="0.20.0",
     description="Tool for installing Lua and LuaRocks locally",
     long_description=long_description,
     keywords="lua",
@@ -29,7 +29,8 @@ setuptools.setup(
         "Programming Language :: Python :: 3.3",
         "Programming Language :: Python :: 3.4",
         "Programming Language :: Python :: 3.5",
-        "Programming Language :: Python :: 3.6"
+        "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.7"
     ],
     py_modules=["hererocks"],
     entry_points={

test/cli_test.py

@@ -53,7 +53,7 @@ class TestCLI(unittest.TestCase):
     def test_install_latest_lua_with_latest_luarocks(self):
         self.assertHererocksSuccess(["--lua", "latest", "--luarocks", "latest"])
         self.assertHererocksSuccess(["--show"], ["Programs installed in", "Compat: default"])
-        self.assertSuccess(["lua", "-v"], ["Lua 5.3.4"])
+        self.assertSuccess(["lua", "-v"], ["Lua 5.3.5"])
         self.assertSuccess(["lua", "-e", "assert(bit32)"])
 
         self.assertSuccess(["luarocks", "--version"])
@@ -146,3 +146,11 @@ class TestCLI(unittest.TestCase):
             "activate 2: {}{}{}".format(path2, os.pathsep, path),
             "deactivate 2: {}".format(path)
         ], from_prefix=False)
+
+    def test_install_lua_5_4_with_luarocks_3(self):
+        self.assertHererocksSuccess(["--lua", "5.4", "--luarocks", "3"])
+        self.assertHererocksSuccess(["--lua", "5.4.0-work1", "--luarocks", "3"])
+
+        if os.name == "nt":
+            self.assertHererocksSuccess(["--lua", "5.4", "--luarocks", "3", "--target", "vs"])
+            self.assertHererocksSuccess(["--lua", "5.4.0-work1", "--luarocks", "3", "--target", "vs"])