Switch GLSLang to upstream (#515)

This CL switches shaderc to use the KhronosGroup glslang instead of the google glslang.
google · Jan 2, 2019 · 355fe6a · 355fe6a
1 parent d6c89a6
commit 355fe6a
Show file tree

Hide file tree

Showing 15 changed files with 356 additions and 156 deletions.
diff --git a/DEPS b/DEPS
@@ -0,0 +1,33 @@
+use_relative_paths = True
+
+vars = {
+  'google_git':  'https://github.com/google',
+  'khronos_git': 'https://github.com/KhronosGroup',
+
+  'effcee_revision' : '8f0a61dc95e0df18c18e0ac56d83b3fa9d2fe90b',
+  'glslang_revision': '83b2647293db3d777e77adbaf04283cf97a18661',
+  'googletest_revision': 'd5932506d6eed73ac80b9bcc47ed723c8c74eb1e',
+  're2_revision': 'f620af75bd693f917c684106d26de1b99ffe0e0d',
+  'spirv_headers_revision': '4618b86e9e4b027a22040732dfee35e399cd2c47',
+  'spirv_tools_revision': '64f2750e5dc553baa2922b780f15049023689ef9',
+}
+
+deps = {
+  'third_party/effcee': vars['google_git'] + '/effcee.git@' +
+      vars['effcee_revision'],
+
+  'third_party/googletest': vars['google_git'] + '/googletest.git@' +
+      vars['googletest_revision'],
+
+  'third_party/glslang': vars['khronos_git'] + '/glslang.git@' +
+      vars['glslang_revision'],
+
+  'third_party/re2': vars['google_git'] + '/re2.git@' +
+      vars['re2_revision'],
+
+  'third_party/spirv-headers': vars['khronos_git'] + '/SPIRV-Headers.git@' +
+      vars['spirv_headers_revision'],
+
+  'third_party/spirv-tools': vars['khronos_git'] + '/SPIRV-Tools.git@' +
+      vars['spirv_tools_revision'],
+}
diff --git a/DEVELOPMENT.howto.md b/DEVELOPMENT.howto.md
@@ -24,7 +24,7 @@ guidelines -- in particular, every file, class, method, data member, and global
 will require a comment.  Reviewers will also expect to see test coverage for
 every code change.  _How much_ coverage will be a judgment call on a
 case-by-case basis, balancing the required effort against the incremental
-benefit.  But coverage will be expected.  As a matter of development philosophy,
+benefit.  Coverage will be expected.  As a matter of development philosophy,
 we will strive to engineer the code to make writing tests easy.
 
 ## Coding style
@@ -50,41 +50,3 @@ likely grow over time.
 | Mac OS X | [![Mac Build Status](https://travis-ci.org/google/shaderc.svg)](https://travis-ci.org/google/shaderc "Mac Build Status") |
 | Windows (x86_64) | [![Windows Build status](https://ci.appveyor.com/api/projects/status/g6c372blna7vnk1l?svg=true)](https://ci.appveyor.com/project/dneto0/shaderc "Windows Build Status") |
 
-
-## glslang
-
-Some Shaderc changes require concomitant changes to glslang.  It is our policy
-to upstream such work to glslang by following the official glslang project's
-procedures.  At the same time, we would like to have those changes available to
-all Shaderc developers immediately upon passing our code review.  Currently this
-is best done by maintaining
-[our own GitHub fork](https://github.com/google/glslang) of glslang, landing
-Shaderc-supporting changes there<sup>\*</sup>, building Shaderc against it, and
-generating pull requests from there to the glslang's original GitHub repository.
-Although a separate repository, this should be treated as essentially a part of
-Shaderc: the Shaderc master should always<sup>\**</sup> build against our
-glslang fork's master.
-
-Changes made to glslang in the course of Shaderc development must build and test
-correctly on their own, independently of Shaderc code, so they don't break other
-users of glslang when sent upstream.  We will periodically upstream the content
-of our fork's `master` branch to the official glslang `master` branch, so all
-the contributions we accept will find their way to glslang.
-
-We aim to keep our fork up to date with the official glslang by pulling their
-changes frequently and merging them into our `master` branch.
-
-<hr><small>
-
-\*: Please note that GitHub uses the term "fork" as a routine part of
-[contributing to another project](https://help.github.com/articles/using-pull-requests/#types-of-collaborative-development-models),
-_not_ in the sense of scary open-source schism.  This is why you'll hear us
-speak of "our fork" and see "forked from KhronosGroup/glslang" atop the
-[google/glslang](https://github.com/google/glslang) GitHub page.  It does _not_
-mean that we're divorcing our glslang development from the original -- quite the
-opposite.  As stated above, we intend to upstream all our work to the original
-glslang repository.
-
-\*\*: with one small exception: if a Shaderc and glslang pull requests need each
-other and are simultaneously cherry-picked, then a `HEAD`s inconsistency will be
-tolerated for the short moment that one has landed while the other hasn't.
diff --git a/Dockerfile b/Dockerfile
@@ -27,13 +27,9 @@ RUN apk add --update \
 
 WORKDIR /root
 RUN git clone https://github.com/google/shaderc
+
 WORKDIR shaderc
-RUN git clone https://github.com/google/googletest.git          third_party/googletest
-RUN git clone https://github.com/google/glslang.git             third_party/glslang
-RUN git clone https://github.com/KhronosGroup/SPIRV-Tools.git   third_party/spirv-tools
-RUN git clone https://github.com/KhronosGroup/SPIRV-Headers.git third_party/spirv-tools/external/spirv-headers
-RUN git clone https://github.com/google/re2.git                 third_party/re2
-RUN git clone https://github.com/google/effcee.git              third_party/effcee
+RUN ./utils/git-sync-deps
 
 WORKDIR build
 RUN cmake -GNinja \

diff --git a/README.md b/README.md
@@ -52,13 +52,6 @@ for more information. See also the [`AUTHORS`](AUTHORS) and
 - `utils/`: utility scripts for Shaderc
 
 Shaderc depends on glslang, the Khronos reference compiler for GLSL.
-Sometimes a change updates both Shaderc and glslang.  In that case the
-glslang change will appear in [google/glslang][google-glslang]
-before it appears upstream in [KhronosGroup/glslang][khr-glslang]
-We intend to upstream all changes to glslang. We maintain the separate
-copy only to stage those changes for review, and to provide something for
-Shaderc to build against in the meantime.  Please see
-[DEVELOPMENT.howto.md](DEVELOPMENT.howto.md) for more details.
 
 Shaderc depends on [SPIRV-Tools][spirv-tools] for assembling, disassembling,
 and transforming SPIR-V binaries.
@@ -79,13 +72,8 @@ branch under the "Artifacts" tab of a certain job.
 
 ```sh
 git clone https://github.com/google/shaderc $SOURCE_DIR
-cd $SOURCE_DIR/third_party
-git clone https://github.com/google/googletest.git
-git clone https://github.com/google/glslang.git
-git clone https://github.com/KhronosGroup/SPIRV-Tools.git spirv-tools
-git clone https://github.com/KhronosGroup/SPIRV-Headers.git spirv-headers
-git clone https://github.com/google/re2.git
-git clone https://github.com/google/effcee.git
+cd $SOURCE_DIR
+./utils/git-sync-deps
 cd $SOURCE_DIR/
 ```
 
@@ -240,7 +228,6 @@ older versions of Shaderc and its dependencies.
 * **Rust:** [shaderc-rs][shaderc-rs]
 
 [khr-glslang]: https://github.com/KhronosGroup/glslang
-[google-glslang]: https://github.com/google/glslang
 [spirv-tools]: https://github.com/KhronosGroup/SPIRV-Tools
 [pyshaderc]: https://github.com/realitix/pyshaderc
 [shaderc-rs]: https://github.com/google/shaderc-rs

diff --git a/glslc/test/option_dash_cap_O.py b/glslc/test/option_dash_cap_O.py
@@ -17,7 +17,7 @@
 from glslc_test_framework import inside_glslc_testsuite
 from placeholder import FileShader
 
-MINIMAL_SHADER = '#version 310 es\nvoid main() {}'
+MINIMAL_SHADER = '#version 310 es\nvoid main() {}\n'
 EMPTY_SHADER_IN_CWD = Directory('.', [File('shader.vert', MINIMAL_SHADER)])
 
 ASSEMBLY_WITH_DEBUG_SOURCE = [
@@ -37,7 +37,8 @@
     '// OpModuleProcessed entry-point main\n',
     '#line 1\n',
     '#version 310 es\n',
-    'void main() {}"\n',
+    'void main() {}\n',
+    '"\n',
     '               OpSourceExtension "GL_GOOGLE_cpp_style_line_directive"\n',
     '               OpSourceExtension "GL_GOOGLE_include_directive"\n',
     '               OpName %main "main"\n',

diff --git a/kokoro/android-release/build.sh b/kokoro/android-release/build.sh
@@ -36,13 +36,10 @@ wget https://dl.google.com/android/repository/android-ndk-r18b-linux-x86_64.zip
 unzip -q android-ndk-r18b-linux-x86_64.zip
 NDK=$PWD/android-ndk-r18b
 
-cd $SRC/third_party
-git clone https://github.com/google/googletest.git
-git clone https://github.com/google/glslang.git
-git clone https://github.com/KhronosGroup/SPIRV-Tools.git spirv-tools
-git clone https://github.com/KhronosGroup/SPIRV-Headers.git spirv-headers
+cd $SRC
+./utils/git-sync-deps
 
-mkdir $SRC/build
+mkdir build
 cd $SRC/build
 
 # Invoke the build.

diff --git a/kokoro/linux/build.sh b/kokoro/linux/build.sh
@@ -69,15 +69,9 @@ wget -q https://github.com/ninja-build/ninja/releases/download/v1.7.2/ninja-linu
 unzip -q ninja-linux.zip
 export PATH="$PWD:$PATH"
 
-cd $SRC/third_party
-git clone https://github.com/google/googletest.git
-git clone https://github.com/google/glslang.git
-git clone https://github.com/KhronosGroup/SPIRV-Tools.git spirv-tools
-git clone https://github.com/KhronosGroup/SPIRV-Headers.git spirv-headers
-git clone https://github.com/google/re2
-git clone https://github.com/google/effcee
-
-cd $SRC/
+cd $SRC
+./utils/git-sync-deps
+
 mkdir build
 cd $SRC/build
 

diff --git a/kokoro/macos/build.sh b/kokoro/macos/build.sh
@@ -31,15 +31,9 @@ unzip -q ninja-mac.zip
 chmod +x ninja
 export PATH="$PWD:$PATH"
 
-cd $SRC/third_party
-git clone https://github.com/google/googletest.git
-git clone https://github.com/google/glslang.git
-git clone https://github.com/KhronosGroup/SPIRV-Tools.git spirv-tools
-git clone https://github.com/KhronosGroup/SPIRV-Headers.git spirv-headers
-git clone https://github.com/google/re2
-git clone https://github.com/google/effcee
+cd $SRC
+./utils/git-sync-deps
 
-cd $SRC/
 mkdir build
 cd $SRC/build
 

diff --git a/kokoro/ndk-build/build.sh b/kokoro/ndk-build/build.sh
@@ -24,14 +24,6 @@ set -x
 BUILD_ROOT=$PWD
 SRC=$PWD/github/shaderc
 
-# Set the glslang repo URL.
-GLSLANG_REMOTE=$1
-GLSLANG_REPO_URL="https://github.com/google/glslang.git"
-if [ $GLSLANG_REMOTE = "KHRONOS" ]
-then
-  GLSLANG_REPO_URL="https://github.com/KhronosGroup/glslang.git"
-fi
-
 # Get NINJA.
 wget -q https://github.com/ninja-build/ninja/releases/download/v1.7.2/ninja-linux.zip
 unzip -q ninja-linux.zip
@@ -41,13 +33,8 @@ export PATH="$PWD:$PATH"
 export ANDROID_NDK=/opt/android-ndk-r15c
 
 # Get shaderc dependencies.
-cd $SRC/third_party
-git clone $GLSLANG_REPO_URL
-git clone https://github.com/google/googletest.git
-git clone https://github.com/KhronosGroup/SPIRV-Tools.git   spirv-tools
-git clone https://github.com/KhronosGroup/SPIRV-Headers.git spirv-headers
-git clone https://github.com/google/re2
-git clone https://github.com/google/effcee
+cd $SRC
+./utils/git-sync-deps
 
 cd $SRC/
 mkdir build

diff --git a/kokoro/ndk-build/build_google.sh b/kokoro/ndk-build/build_google.sh
diff --git a/kokoro/ndk-build/build_khronos.sh b/kokoro/ndk-build/build_khronos.sh
@@ -22,4 +22,4 @@ set -e
 set -x
 
 SCRIPT_DIR=`dirname "$BASH_SOURCE"`
-source $SCRIPT_DIR/build.sh KHRONOS
+source $SCRIPT_DIR/build.sh
diff --git a/kokoro/ndk-build/continuous_google.cfg b/kokoro/ndk-build/continuous_google.cfg
diff --git a/kokoro/windows/build.bat b/kokoro/windows/build.bat
@@ -24,22 +24,11 @@ set VS_VERSION=%2
 :: Force usage of python 2.7 rather than 3.6
 set PATH=C:\python27;%PATH%
 
-cd %SRC%\third_party
-git clone https://github.com/google/googletest.git
-git clone https://github.com/google/glslang.git
-git clone https://github.com/KhronosGroup/SPIRV-Tools.git spirv-tools
-git clone https://github.com/KhronosGroup/SPIRV-Headers.git spirv-headers
-git clone https://github.com/google/re2
-git clone https://github.com/google/effcee
+cd %SRC%
+python utils\git-sync-deps
 
 cmake --version
 
-:: If you need to pin googletest to a specific revision, uncomment
-:: the following two lines (and set the commit hash).
-:: cd googletest
-:: git checkout e93da23920e5b6887d6a6a291c3a59f83f5b579e
-
-cd %SRC%
 mkdir build
 cd %SRC%\build
 

diff --git a/shader.vert.spvasm b/shader.vert.spvasm
@@ -0,0 +1,27 @@
+; SPIR-V
+; Version: 1.0
+; Generator: Google Shaderc over Glslang; 7
+; Bound: 7
+; Schema: 0
+               OpCapability Shader
+          %2 = OpExtInstImport "GLSL.std.450"
+               OpMemoryModel Logical GLSL450
+               OpEntryPoint Vertex %main "main"
+          %1 = OpString "../shader.vert"
+               OpSource ESSL 310 %1 "// OpModuleProcessed entry-point main
+// OpModuleProcessed client vulkan100
+// OpModuleProcessed target-env vulkan1.0
+// OpModuleProcessed entry-point main
+#line 1
+#version 310 es
+void main() {}
+"
+               OpSourceExtension "GL_GOOGLE_cpp_style_line_directive"
+               OpSourceExtension "GL_GOOGLE_include_directive"
+               OpName %main "main"
+       %void = OpTypeVoid
+          %4 = OpTypeFunction %void
+       %main = OpFunction %void None %4
+          %6 = OpLabel
+               OpReturn
+               OpFunctionEnd