23737 reports that flags generated by pkg-config -libs are being checked against the compiler whitelist, and CGO_CFLAGS_ALLOW, when they should instead by checked against the linker whitelist and CGO_LDFLAGS_ALLOW. There is a CL to fix this: https://golang.org/cl/92755.

The linker whitelist should accept .a files as it already accepts .o, .so, etc., files. There is a CL to fix this: https://golang.org/cl/92855. This is #23739.

Comments on #23672 list these compiler options:

-fno-rtti
-fpermissive

and these linker options:

-Wl,-framework
-Wl,--no-as-needed

23742 suggests adding the compiler option -fmodules. The clang compiler supports a number of -fmodules options, but it's not clear that they are all safe. In particular -fmodules-cache-path and -fmodules-user-build-path would seem to permit specifying a path that clang will use to read modules, which could perhaps change the compilation mode in various ways.

23743 suggests adding the linker option -Wl,--no-as-needed. There is a CL for this: https://golang.org/cl/92795.

23744 suggests adding these compiler options:

-finput-charset=UTF-8
--std=c99
-pedantic-errors

There are many compiler and linker options that may be used with either a single dash or a double dash. We should be similarly lax in the whitelists.

For orthogonality: I forget if the option to add a directory to -framework's search path is already covered or not. I also forget what option that is. (The typical use case that I can think of is /Library/Frameworks, which is where Apple puts app-specific frameworks, and is not searched by default.)

Also is -as-needed safe to use with cgo in the first place? This blog post (which is the first result I can find for "gcc as-needed") says that it's a positional argument, but I'm not sure if cgo guarantees anything about where your flags go on the resultant command lines.

@andlabs It's fine to write

#cgo LDFLAGS: -Wl,--as-needed -loptlib -WL,--no-as-needed

In any case, the topic for this issue should be whether options are safe to use from go get.

When using:

#cgo !windows pkg-config: --static ${SRCDIR}/vendor/libgit2/build/libgit2.pc

compilation fails with the following message:

invalid pkg-config package name: --static

Looking over the code (for go 1.9.4) it appears there aren't any enviroment variables that can be used to whiteslist pkg-config arguments.

@rgburke pkg-config output goes through the same FLAGS_ALLOW variables as other outputs.

However, it appears that pkg-config -libs checks CGO_CFLAGS_ALLOW when it ought to check CGO_LDFLAGS_ALLOW.

We link a large number of C libraries statically in a closed-source project. Up until now we have been doing the following:

#cgo LDFLAGS:/path/to/one/liblibrary1.a
#cgo LDFLAGS:/path/to/two/liblibrary2.a
etc.

This is of course disallowed now. A workaround:

#cgo LDFLAGS:-L/path/to/one -llibrary1
#cgo LDFLAGS:-L/path/to/two -llibrary2

However some of these directories contain dynamic libraries too, which confuses the linker. Another option is to add '/' to the list of allowed "names" in https://go-review.googlesource.com/c/go/+/92855 In particular change on line 91 is:

re(`[a-zA-Z0-9_\/].*\.(o|obj|dll|dylib|so|a)`), // direct linker inputs: x.o or libfoo.so (but not -foo.o or @foo.o)

the latter option fixes our issue but I can't speak to its impact on security.

@mirtchovski there's a patch for that (the issue is that .a was not whitelisted, but other object file formats were)

.a is now whitelisted (after that patch) so 'libsomething.a' will work, but '/path/to/libsomething.a' will not work.

@ianlancetaylor @rgburke I actually ran into the very same problem with --static which lead me down the hole to #23737. --static is rejected because it doesn't appear to be a valid package name _before_ attempting to execute pkg-config, here https://github.com/golang/go/blob/104445e3140f4468839db49a25cb0182f7923174/src/cmd/go/internal/work/exec.go#L939-L940.

Our quick fix internally was to point PKG_CONFIG to a script which simply executed pkg-config --static "$@".

@jeddenlea - Thank you very much! I was trying to find a workaround but didn't think of that.

We hit this with -msse and -msse4.2.

I hit this issue with "${SRCDIR}/file.o" in cgo LDFLAGS.

I'd like to argue that we should allow plain filenames that is linker input
files in LDFLAGS
(at least *.a, *.o and *.so).

Unlike .a and .so which in theory could be handled with "-L${SRCDIR}
-lname", adding
extra object files to linker command cannot be fixed that way.

A workaround is to set CGO_LDFLAGS_ALLOW, but that's very cumbersome.
Another
alternative is to rename my file.o into file.syso, however, for my specific
case, I can't use
this option because I only include that file.o when a specific build tag is
included (the
build tag applies to file that contains the #cgo LDFLAGS preamble) and
there is no way
to set arbitrary build tag in syso filenames.

If we ignore previous deployed Go versions, we might be able to include a
new
"#cgo LDLIBS" that is designed specifically for adding more files to linker
command line.
Then we can have a stricter rule for LDLIBS (filenames only, and no dash
allowed in prefix.)

We've hit this with --std=c99

-std=c++11

I guess -std= should be white listed?

Probably on whitelist: -fopenmp

Error when building bimg package using Go v1.10rc2,

23:24 ~/go-test
master ms 130 % go get -v github.com/h2non/bimg
github.com/h2non/bimg (download)
github.com/h2non/bimg
go build github.com/h2non/bimg: invalid flag in pkg-config --cflags: -fopenmp

I'm unable to whitelist -isystem as the argument next to it (which is a path) will be denied

We also needed to use this workaround: https://github.com/golang/go/issues/23749#issuecomment-364239496

We previously had:

#cgo linux LDFLAGS: ${SRCDIR}/../../../../path/to/thing/libthing_static.a

and have shifted to:

#cgo linux LDFLAGS: -L${SRCDIR}/../../../../path/to/thing -lthing_static

The .../path/to/thing is outside ${SRCDIR}.

Adding this comment so this issue includes an example with a libthing.a reference that needs SRCDIR expansion to resolve.

on OSX, with the newly minted go1.9.4 with its cgo flag restrictions, I was telling the linker specifically what .a archive to link against: (here https://github.com/gijit/gi/blob/master/vendor/github.com/glycerine/golua/lua/lua.go#L10 )

~~~

cgo LDFLAGS: ${SRCDIR}/../../../LuaJIT/LuaJIT/src/libluajit.a -lm -ldl

~~~

produces on attempt to build:

~
make build
go build -ldflags "-X main.LastGitCommitHash=30259813c10c0f6b63768b4f35358828e2e29f0b -X main.BuildTimeStamp=2018-02-09T22:49:48+0700 -X main.GitBranch=master -X main.NearestGitTag=v0.9.6 -X main.GoVersion=go_version_go1.9.4_darwin/amd64" -o gi
go build github.com/gijit/gi/vendor/github.com/glycerine/golua/lua: invalid flag in #cgo LDFLAGS: /Users/jaten/go/src/github.com/gijit/gi/vendor/github.com/glycerine/golua/lua/../../../LuaJIT/LuaJIT/src/libluajit.a
make[2]: * [build] Error 1
make[1]: [install] Error 2
make: ** [install] Error 2
jaten@jatens-MacBook-Pro ~/go/src/github.com/gijit/gi (master) $
~
I tried the -L -l workaround,
~~~

cgo LDFLAGS: -L${SRCDIR}/../../../LuaJIT/LuaJIT/src -lluajit -lm -ldl

~~~
but then the linker thinks it can use a different library by the same name in a different location, and I get a runtime rather than linktime error.

~~~
at runtime...
dyld: lazy symbol binding failed: Symbol not found: _luajit_ctypeid
Referenced from: /var/folders/6s/zdc0hvvx7kqcglg5yqm3kl4r0000gn/T/go-build615587282/github.com/gijit/gi/pkg/compiler/_test/compiler.test
Expected in: /usr/local/lib/libluajit-5.1.2.dylib

dyld: Symbol not found: _luajit_ctypeid
Referenced from: /var/folders/6s/zdc0hvvx7kqcglg5yqm3kl4r0000gn/T/go-build615587282/github.com/gijit/gi/pkg/compiler/_test/compiler.test
Expected in: /usr/local/lib/libluajit-5.1.2.dylib

SIGTRAP: trace trap
PC=0x7fff66ff4075 m=0 sigcode=1
signal arrived during cgo execution
~~~
/usr/local/lib/libluajit-5.1.2.dylib is not the library required, though that was dynamically linked; rather it must be the one specified in ${SRCDIR}/../../../LuaJIT/LuaJIT/src/libluajit.a.

So still searching for a workaround.

Update: looks like adding this to my makefiles does the trick.
~
export CGO_LDFLAGS_ALLOW="${GOPATH}/src/github.com/gijit/gi/vendor/github.com/glycerine
/golua/lua/../../../LuaJIT/LuaJIT/src/libluajit.a";
~

Update: Thankfully, Ian pointed out that the shorter regex version will do:
~
export CGO_LDFLAGS_ALLOW=".*.a";
~

What is -Wl,-framework for? If that's the Apple framework, it has an argument, so you'd probably want -Wl,-framework,foo. But if it's the Apple framework then -framework (no -Wl,) works just as well.

With 1.10rc2, golang.org/x/net/internal/socket doesn't build any more for Solaris.

$ GOOS=solaris go build golang.org/x/net/ipv4
# golang.org/x/net/internal/socket
ext/src/golang.org/x/net/internal/socket/sys_solaris.go:24:3: //go:cgo_import_dynamic libc___xnet_getsockopt __xnet_getsockopt "libsocket.so" only allowed in cgo-generated code
ext/src/golang.org/x/net/internal/socket/sys_solaris.go:25:3: //go:cgo_import_dynamic libc_setsockopt setsockopt "libsocket.so" only allowed in cgo-generated code
ext/src/golang.org/x/net/internal/socket/sys_solaris.go:26:3: //go:cgo_import_dynamic libc___xnet_recvmsg __xnet_recvmsg "libsocket.so" only allowed in cgo-generated code
ext/src/golang.org/x/net/internal/socket/sys_solaris.go:27:3: //go:cgo_import_dynamic libc___xnet_sendmsg __xnet_sendmsg "libsocket.so" only allowed in cgo-generated code

(There's no cgo happening here as this is a cross build, but I guess that doesn't matter.)

I can't link to a static lib file by specifying a full path to it:

#cgo LDFLAGS: /usr/local/lib/libsecp256k1.a

I see no workaround for it :(

@piotrnar CGO_LDFLAGS_ALLOW='.*\.a$'

@ianlancetaylor, thank you!

that will do for me, but telling all the other people who use my project to do CGO_LDFLAGS_ALLOW='.*\.a$' for go 1.9.4 seems a bit dodgy.

hopefully there will be a better (out of the box) solution in the future.

@piotrnar Yes, the point of this issue is to collect all the changes we need to make to the whitelist. Certainly we will be whitelisting .a files.

can you please add also -fstack-protector?

thanks :)

Whitelist -static-libstdc++ please.

The package github.com/flynn/hid fails to build with 1.9.4 due to passing -fconstant-cfstrings via LDFLAGS on darwin.

Please add these linker flags to the white list
-Wl,-Bstatic
-Wl,-Bdynamic
-Wl,--start-group
-Wl,--end-group

Honestly: At this point a blacklist of bad options seems more useful than such an extensive whitelist to me.

Going forward, a blacklist would mean that if some new potentially-insecure compiler option were introduced, all Go releases would become vulnerable to that option. To the extent that we can protect against this kind of attack at all, I think it has to be a whitelist.

some new potentially-insecure compiler option were introduced

I still think blacklist is preferable. Because it cuts both ways. If any new compiler option can't be utilized until it is whitelisted, that would seem to require a new Go release every time any new C compiler adds a flag...

@glycerine That's why we provide the environment variables as an escape hatch. You can always use the environment variables until the whitelist is updated.

The problem is that env variables don't work for projects that are installed purely through 'go get'.

Another approach: allow the top level, named go project to set environment variables.

Then the primary 'go build' project being installed can whitelist the flags it needs, e.g. using the ALLOW regex, while still preventing dependent projects from doing aribitrary things.

@glycerine I'm not sure I understand how that would work. But I suggest that you open a separate issue for that suggestion, rather than discussing it on this issue, which is intended to collect options that need to be whitelisted.

For now we've decided on implementing a whitelist. That may change in the future but this issue is not the place to discuss it (feel free to file a new one). We are trying to collect options that should be whitelisted here and nothing more.

Thank you.

invalid flag in #cgo CFLAGS: -pipe

go build github.com/zchee/docker-machine-driver-xhyve/vendor/github.com/zchee/libhyperkit: invalid flag in #cgo CFLAGS: -fno-common

Hi, please add these flags to the whitelist, -Wl,--enable-new-dtags

Can't build therecipe/qt

go build github.com/therecipe/qt/core: invalid flag in #cgo CFLAGS: -pipe

Adding .*\.a to the allowed flags would be great.
See https://github.com/golang/go/issues/23807

--mms-bitfields also seems be required.

Isn't it reasonable to assume that every compiler (and linker, etc) option exists for a reason, and the whitelist should cover all of them except the ones specifically deemed dangerous? Obviously there's hundreds of them and the maintenance of the white list isn't going to be fun, but if that's the path that's been decided on...

Isn't it reasonable to assume that every compiler (and linker, etc) option exists for a reason, and the whitelist should cover all of them except the ones specifically deemed dangerous? Obviously there's hundreds of them and the maintenance of the white list isn't going to be fun, but if that's the path that's been decided on...

This has probably been discussed internally but I found the thing surprising for a patch release: I would have expected the offending flags to have been blacklisted first to plug the compiler plugin hole, and a whitelist system enabled for 1.10, whose list could have been built up during RC. The extra env vars are not quite practical to integrate in some build systems, and I observed this leads to people just reverting to 1.9.3 and thus being completely unprotected, which I believe is entirely counterproductive.

At which point does a whitelist full of wildcards and regexes becomes a blacklist in disguise?

gomobile uses the flags -fobjc-arc -fmodules -fblocks:

https://github.com/golang/mobile/blob/5704e182c7003d4b7e94c23373f3fad4e5ceb25a/bind/genobjcw.go#L319

https://github.com/augustoroman/v8 uses -pthread:

https://github.com/augustoroman/v8/blob/master/v8.go#L15

-flto

Could you put an alphabetically sorted list of the "approved" ones at the top of this issue for overview until the next update is out? Whoever submits the CL may appreciate that as well.

Right now (especially since this issue exists) I'm more concerned about what's slipping through the cracks: what build flags people have removed from their projects without consulting us first, giving them the misguided belief that cgo (and by extension Go) is broken and buggy and has regressions and its developers don't know what they are doing. :S (Or worse, you don't know what you are doing and are obviously building my package wrong, because you are the one missing dependency xyz!)

Some of the "referenced this issue" links list more blacklisted flags that aren't in this issue proper yet. I also vouch for having a master list at the top, especially to ward off duplicates. People are still running into the static archive issue...

This makes me wonder if gcc and clang themselves could, or even should, provide an option --no-unsafe-options that a) would do the dirty work for us b) be resilient to future changes and c) cannot be turned off by another option (once it's there, it's there, period). Or is go get the first and only situation where this kind of filtering is needed?

If we insist on a whitelist, then I don't know why there needs to be any delay waiting for input.

The compilers document their flags. As someone said above, each flags is there for a reason.

Sampling by waiting for user input on usage will be under-representative, unreliable, and produce painful results for all of us who in the future find need for a flag that wasn't realized or sampled now.

Moreover it seems straightforward to derive the white list by this procedure, in pseudo code:

List all the C/C++ compilers and versions of each compiler supported;
For each compiler-version, list all the flags available from their docs;
For each flag, decide to put it on the whitelist or the (invisible) blacklist.

Add whitelisting code for all the flags on the accumulated whitelist.

I still think this is crazy (verging on reductio ad absurdum) compared to blacklisting the one (or two) vulnerable flags. But in all seriousness, if we're going to insist on a whitelist, the algorithm above is correct, eliminates the guesswork, and can be executed without delay. Perhaps the only additional user input required would be to establish the range of compilers/versions to be supported.

The only options that matter here are the ones that are reasonable to put on a #cgo CFLAGS or #cgo LDFLAGS line in a .go file, or are reasonable to output from a call to pkg-config --cflags or pkg-config --libs. That is a small subset of the total number of compiler options.

You also vastly underestimate how many options gcc and clang even have. I doubt they are all documented, even in the source code — I wouldn't be surprised if some are generated at runtime! Also there might be options that are target triple-dependent... (This is also why I say that a canonical list of safe flags would need to be maintained by the gcc and clang devs.)

I can't say anything about the delay in patching individual flags, though.

@anacrolix Over in #23672 you suggested that we need to whitelist -Wl,-framework. But -framework normally takes an option. Can you show the exact use case? Thanks.

You also vastly underestimate how many options gcc and clang even have

@andlabs Ian is a gcc developer. I'm sure he's well aware.

Change https://golang.org/cl/93836 mentions this issue: cmd/go: add options to security whitelist

I sent a CL that I think covers all of the above options: https://golang.org/cl/93836.

Please let me know if you see any problems with it or if you know of any options people are using that it does not cover. Thanks.

From LLVM Go bindings point of view, following linker options should be in the white list.

• -Wl,-search_paths_first
• -Wl,-headerpad_max_install_names

Linker options are generated by command llvm-config --ldflags and they are in the output.

ref: https://reviews.llvm.org/D43070

@magical I was responding to @glycerine there =P

@ianlancetaylor also apologies if I am making you feel rushed with this

@andlabs Ah, sorry

@ianlancetaylor With that CL I still cannot build golang.org/x/net/internal/socket for Solaris. It's not obvious to me from the error what flags I need to ask for.

$ ./bin/go version
go version devel +09dc376990 Tue Feb 13 20:58:04 2018 -0800 darwin/amd64
$ GOOS=solaris ./bin/go get -u golang.org/x/net/ipv4
# golang.org/x/net/internal/socket
../../ext/src/golang.org/x/net/internal/socket/sys_solaris.go:24:3: //go:cgo_import_dynamic libc___xnet_getsockopt __xnet_getsockopt "libsocket.so" only allowed in cgo-generated code
../../ext/src/golang.org/x/net/internal/socket/sys_solaris.go:25:3: //go:cgo_import_dynamic libc_setsockopt setsockopt "libsocket.so" only allowed in cgo-generated code
../../ext/src/golang.org/x/net/internal/socket/sys_solaris.go:26:3: //go:cgo_import_dynamic libc___xnet_recvmsg __xnet_recvmsg "libsocket.so" only allowed in cgo-generated code
../../ext/src/golang.org/x/net/internal/socket/sys_solaris.go:27:3: //go:cgo_import_dynamic libc___xnet_sendmsg __xnet_sendmsg "libsocket.so" only allowed in cgo-generated code

@calmh Yes. I think we're going to have to fix that one with changes to golang.org/x/net. It is already relying on many internal details, and I think we need to change the way it does so.

@calmh Would you be able to test https://golang.org/cl/94015 on an actual Solaris system?

Change https://golang.org/cl/94015 mentions this issue: internal/socket: rework Solaris reliance on syscall package

@rsc @ianlancetaylor @anacrolix I figured out where -Wl,-framework is coming from: it's coming from the pkg-config file for GLib, which is a dependency of GTK+. Specifically, it includes the CoreFoundation link for internationalization purposes.

Specific references: https://gitlab.gnome.org/GNOME/glib/blob/master/glib-2.0.pc.in#L14 where @INTLLIBS@ gets -Wl,-framework -Wl,CoreFoundation from https://gitlab.gnome.org/GNOME/glib/blob/master/m4macros/glib-gettext.m4#L143

There are other -Wl,-framework instances in various other pkg-config files; some are part of the projects themselves (as above), and some are injected by Homebrew. On my system right now, I see libcdio and SDL both use -Wl,-framework,FrameworkName. (Which means, yes, both -Wl,-framework -Wl,FrameworkName and -Wl,-framework,FrameworkName are found in pkg-config files. Go figure.)

Change https://golang.org/cl/94018 mentions this issue: cmd/compile: permit go:cgo_import_dynamic anywhere

@calmh Trying a different approach in https://golang.org/cl/94018.

@andlabs Thanks, added to CL 93836.

I'm static linking SDL and the pgkconfig contains -Wl,--no-undefined as well as preprocessor defines.

I need you to add the flags below the white list, so they can compile projects that use the cgoemitter package

go build github.com/supermock/cgoemitter-demo/x: invalid flag in #cgo LDFLAGS: -Wl,-unresolved-symbols=ignore-all

Thank you very much in advance

Update CL again.

I've added a wiki page to describe this problem at https://golang.org/wiki/InvalidFlag, and I'm updating the CL so that the error message points people to the wiki.

Change https://golang.org/cl/94158 mentions this issue: doc: add note about invalid flag errors to 1.10 release notes

invalid flag in #cgo LDFLAGS: -O3

Change https://golang.org/cl/94655 mentions this issue: [release-branch.go1.10] doc: add note about invalid flag errors to 1.10 release notes

Change https://golang.org/cl/94675 mentions this issue: [release-branch.go1.10] cmd/compile: permit go:cgo_import_dynamic anywhere

Change https://golang.org/cl/94676 mentions this issue: [release-branch.go1.10] cmd/go: add options to security whitelist

Also receiving this issue, if the following can be added

go build github.com/andlabs/ui: invalid flag in #cgo LDFLAGS: -mmacosx-version-min=10.8

@bgk- The patch that has been applied addresses that. If upgrading to 1.10 isn't feasible right now, I'd have to suggest waiting to see if a 1.9.5 happens.

Should 1.10 have fixed the invalid pkg-config package name: --static issue? I just pulled down the latest version from homebrew and I'm still seeing it:

➜  ~ go version
go version go1.10 darwin/amd64
... 
➜  ~ go build
...
go build github.com/flier/gohs/hyperscan: invalid pkg-config package name: --static

@ptoomey3 See #23875

Reopening for 1.9.5.

I will avoid the X Y problem and rather explain fully what I'm doing. I'm creating an postgresql extension in go with -buildmode=c-shared.

In the documentation for creating C extensions for postgresql is this as the way to build the shared object:

cc -fPIC -c foo.c
cc -shared -o foo.so foo.o

So I've added #cgo LDFLAGS: -shared to my source code. It worked until recently (go1.10) when it stops building with:

invalid flag in #cgo LDFLAGS: -shared

Just grabbed go 1.10, ran into this:

invalid flag in #cgo LDFLAGS: -Wl,-static

This is a gcc linker flag to force static linking for anything after that option on the link line.

@spackard For the record, that is not what that option means. You are thinking of -Bstatic. The -static option directs the linker to not search for any shared libraries to satisfy any -l options. -static is not a positional option; it doesn't matter where it appears on the command line.

@ianlancetaylor You're correct about the positional. It is how we're forcing static linking regardless. For the record, adding it to CGO_LDFLAGS_ALLOW does work.

I am having the same issue

invalid flag in #cgo CFLAGS: -m32

We hit this with -O3, -static-libgcc and -static-libstdc++

-O3:

cgo windows LDFLAGS: -O3 -L./ -lmass -static-libgcc -static-libstdc++

cgo linux LDFLAGS: -O3 -L./ -lmass -ldl -lm -lrt -static-libgcc -static-libstdc++

cgo CFLAGS: -O3

invalid flag in #cgo LDFLAGS: -O3

-static-libgcc:

cgo windows LDFLAGS: -L./ -lmass -static-libgcc -static-libstdc++

cgo linux LDFLAGS: -L./ -lmass -ldl -lm -lrt -static-libgcc -static-libstdc++

invalid flag in #cgo LDFLAGS: -static-libgcc

-static-libstdc++:

cgo windows LDFLAGS: -L./ -lmass -static-libstdc++

cgo linux LDFLAGS: -L./ -lmass -ldl -lm -lrt -static-libstdc++

invalid flag in #cgo LDFLAGS: -static-libstdc++

using hardcoded static whitelists is a bad idea, maybe we should use an config file with some default settings like

$GOROOT/bin/cgo.cfg
[whitelists]
blablabla
...

so we can do some customize

@kruglinski You can customize by using the CGO_CFLAGS_ALLOW environment variable and friends.

@ianlancetaylor

Sorry, I just missed about this, :-) Good to know, many thinks!

24124 reports the following linker option:

-Wl,-z,relro

Linker option -Wl,--subsystem,windows and compiler option -mwindows are both missing.

I'm trying to make gomobile/gobind generate standalone bindings. Some of that work involves converting CGO_*FLAGS environment variables to #cgo directives, which uncovered a few more flags:

````
-isysroot (ios)
-mios-simulator-version-min= (ios)
-miphoneos-version-min= (ios)

-target (for android)
--sysroot (for android)
-gcc-toolchain (for android)
````

I believe all but the last, -gcc-toolchain, are safe to add to the whitelist.

Following flags are needed to build Go bindings for Godot game engine without CGO_LDFLAGS_ALLOW on macOS:

-Wl,-undefined,dynamic_lookup

More CFLAGS from zchee/libhyperkit:

• -fmessage-length=152
• -fdiagnostics-show-note-include-stack
• -fmacro-backtrace-limit=0

v8worker needs -stdlib=libstdc++ on Mac OS X.

CL 103156 OK for Go 1.9.5

Change https://golang.org/cl/103135 mentions this issue: [release-branch.go1.9] cmd/go: add options to security whitelist

I couldn't see it on the list, could libtool also be whitelisted

invalid flag in #cgo LDFLAGS: -I/usr/local/share/libtool

@PassKit -I is a cflag (which is in the list) not a ldflag, why are you putting it in ldflags?

Re-opening to collect more whitelist requests.

From #24703

CFLAGS: -fno-plt

At this point I think it's OK to decide that we will only update the 1.9 and 1.10 release branches if some popular package to fail to build and that has somehow not been reported yet. I don't think we need to keep backporting every random option. We can just add them to tip for 1.11. Whether we track them in this issue or elsewhere, I don't care.

sgtm

@AlexRouSg my issue was related to this library, which recommended whitelisting "-I" as an ldflag, which I am currently using as a workaround. https://github.com/miekg/pkcs11/issues/63

Sorry for being late, but it would be great if these would be whitelisted in 1.9.5 or 1.10.2

CXXFLAGS: -F/Users/user/Qt/5.10.1/clang_64/lib
CFLAGS: --sysroot=/Users/user/android-ndk-r14b/sysroot
CFLAGS: -mfloat-abi=softfp
CFLAGS: -fno-builtin-memmove
CFLAGS: -mthumb
CFLAGS: -fobjc-nonfragile-abi
CFLAGS: -fobjc-legacy-dispatch
CFLAGS: -fno-keep-inline-dllexport
CXXFLAGS: -mthreads
CFLAGS: -Wp,-D_FORTIFY_SOURCE=2
CFLAGS: --param=ssp-buffer-size=4
CFLAGS: -mfloat-abi=hard
CFLAGS: -fvisibility-inlines-hidden
CFLAGS: -mfpmath=sse
CFLAGS: -fasynchronous-unwind-tables
CFLAGS: -feliminate-unused-debug-types
CFLAGS: -marm
CFLAGS: -mabi=aapcs-linux
CFLAGS: -mthumb-interwork

and

LDFLAGS: -headerpad_max_install_names
LDFLAGS: -Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk
LDFLAGS: -Wl,-rpath,@executable_path/Frameworks
LDFLAGS: --sysroot=/Users/user/android-ndk-r14b/platforms/android-16/arch-arm/
LDFLAGS: -Wl,-rpath-link=/Users/user/Qt/5.10.1/android_armv7/lib
LDFLAGS: -Wl,--allow-shlib-undefined
LDFLAGS: -Wl,-e,_qt_main_wrapper
LDFLAGS: -Wl,-O1
LDFLAGS: -Wl,-rpath-link,/opt/Qt/5.10.0/gcc_64/lib
LDFLAGS: -Wl,-s
LDFLAGS: -Wl,-subsystem,console
LDFLAGS: -mthreads
LDFLAGS: -rdynamic
LDFLAGS: -mfloat-abi=hard

Thanks in advance.

@therecipe We can add those to 1.11. But for 1.9.5 or 1.10.2: what package(s) need these?

@ianlancetaylor Those are all needed for the different targets of https://github.com/therecipe/qt
It's okay if you won't whitelist them all at once, but please let me know which one I would need to take care myself of.

I am going to guess it's their own Qt package, which I know is old enough to be well established; I'm not fully sure if it is (someone else will need to say).

That being said:

• Aren't some of the LDFLAGS already whitelisted without the -Wl, prefix?
• Does -e have an effect in Go?
• Wasn't -F already whitelisted? That turns out to be the option I was referring to above.
• (To @therecipe) Why are you guarding a -D with a -Wp? Is there a reason to have a macro defined but only in the preprocessor? I don't know what kind of voodoo Qt does here... Or is this a recommended list of options provided by Qt itself?
• (To @ianlancetaylor) I can sort of understand not backporting new options to 1.9 (I'm ambivalent on the issue because I don't have Go version usage metrics), but why not 1.10 until 1.12 is released?
• Regardless of the answer to the above, I wonder if backporting the ARM ABI options listed there is worth doing...

@andlabs Only because we have to stop sometime, so why not stop now? But I'm OK with continuing to backport patches if it seems sufficiently useful.

@andlabs Yes, those are all recommended flags. I didn't manually add a single one of them myself.
-Wp,-D_FORTIFY_SOURCE=2 was from the sailfish target iirc, I don't know why they did that.

@therecipe in that case, where do Qt and Sailfish make those suggestions (that is, is there a web page or CLI tool that gives them to you)? I'm curious now.

In the meantime, while that list is addressed by the Go team, I still wonder how many of your LDFLAGS warnings go away if you remove -Wl, prefixes; try that and see? Same for the -Wp, prefix in the one CFLAGS. I don't know if this will have adverse effects on the build, but it shouldn't in an ideal world because that's what LDFLAGS is for... (You'll also have to change the commas to spaces. These -Wx, options basically tell gcc and clang to send the arguments directly to the linker (or assembler or preprocessor) instead of handling them themselves; if I am guessing correctly the idea is for OS-specific options that aren't yet provided by gcc and clang directly, but I don't know for sure...)

@andlabs Yeah, one of Qt's build tools qmake consumes *.pro, *.spec (mkspec), *.conf and a lot of other formats to generate regular Makefiles. I just feed it some dummy *.pro files and extract the flags from the Makefiles. That way, I don't really have to resolve dependencies myself, the build process can work easier with 3rd-party libs, I don't have to manually manage the c or ld flags and untested targets or newer/older versions of Qt work out of the box most of the time. I just occasionally have to blacklist some flags that cause trouble, but that's very rare.

I tried to find the sailfish mkspec that pulls in -Wp,-D_FORTIFY_SOURCE=2, but it's probably buried somewhere in the mersdk or so. But here is the official list of targets TQtC maintains support for: https://github.com/qt/qtbase/tree/5.11/mkspecs and there are a lot of unofficial ones maintained by independent parties. That's why I really don't want to manually mess with the flags.

I can whitelist them in the build tool used by the binding (which wraps go) in meantime, but the regular Go user who just wants to use go build ... will sooner or later raise an issue ... (at least for the desktop targets)

-ffast-math

The official SAP HANA client driver (the cgo based shared library) comes with #cgo LDFLAGS: -Wl,-rpath -Wl,\$ORIGIN which results in go build SAP/go-hdb/driver: invalid flag in #cgo LDFLAGS: -Wl,-rpath. Also see SAPs documentation at https://help.sap.com/viewer/0eec0d68141541d1b07893a39944924e/2.0.03/en-US/fba20e31f75c4f7ca5629083869069e5.html?q=golang%20driver for details.

@cbgo see #23672

Code that passes a flag and argument pair to the linker must now use one -Wl flag instead of two: use #cgo LDFLAGS: -Wl,-rpath,$ORIGIN, not #cgo LDFLAGS: -Wl,-rpath -Wl,$ORIGIN.

@AlexRouSg thanks a lot. I should have read that more carefully :-)

@PassKit Did you solve this problem？

@zcm211 If you're talking about https://github.com/miekg/pkcs11 they removed the -I... linker flag in feb. If you're using a package that uses it then you have to set the environmental variable CGO_LDFLAGS_ALLOW="-I.*"

darwin 64-bit, go1.10.2, I thought .o files were whitelisted for LDFLAGs, but I'm getting:
~
jaten@jatens-MacBook-Pro ~/go/src/github.com/go-interpreter/chezgo (master) $ make run
cd chez_scheme_9.5.1/c; make
make[1]: Nothing to be done for `doit'.
go build && ./chezgo
go build github.com/go-interpreter/chezgo: invalid flag in #cgo LDFLAGS: ./chez_scheme_9.5.1/boot/a6osx/kernel.o
make: * [run] Error 1
jaten@jatens-MacBook-Pro ~/go/src/github.com/go-interpreter/chezgo (master) $ go version
go version go1.10.2 darwin/amd64
jaten@jatens-MacBook-Pro ~/go/src/github.com/go-interpreter/chezgo (master) $
~
due to the cgo preamble in https://github.com/go-interpreter/chezgo/blob/master/embed.go#L10
~~~

cgo LDFLAGS: -liconv -lm -lncurses -L/usr/local/lib ./chez_scheme_9.5.1/boot/a6osx/kernel.o

~~~

@glycerine According to https://go-review.googlesource.com/c/go/+/94676 they are. Maybe try a full path instead of a relative one?

Good catch @AlexRouSg, the accept regex is buggy as you point out;
~
re([a-zA-Z0-9_/].*\.(a|o|obj|dll|dylib|so)), // direct linker inputs: x.o or libfoo.so (but not -foo.o or @foo.o)
~
src/cmd/go/internal/work/security.go#121 should allow .o files to start with a . to support relative paths.

After all, I can't predict the users' GOPATH, or how nested the location of that file will become, so setting an absolute path isn't practical.

Is the .o file in the source directory? If so, referring to the source directory is what ${SRCDIR} was provided for. (I forget specifically why this was introduced, but it was not because of this issue.)

@andlabs Even if there are klunky workarounds, relative paths should be linkable, and that is clearly a bug.

It is, except IIRC there is no guarantee that the link would be relative TO the source directory (it could very well be in $WORK, and then your relative path breaks again)... Again, I've forgotten the history; someone else will need to explain.

gtk4 uses -mfpmath=sse

@ianlancetaylor Instead of having separate whitelists for cflags and ldflags, should there just be one list? gcc/llvm doesn't seem to care that you mix ldflags into cflags and vice versa. And it seems sometimes C devs do that causing issues like #25493 or https://github.com/golang/go/issues/23749#issuecomment-379969818

Ah, I see we have a ticket already, so let me mention "-D_THREAD_SAFE" from protobuf as documented in #25493

Change https://golang.org/cl/115415 mentions this issue: cmd/go: accept more safe CFLAGS/LDFLAGS

Change https://golang.org/cl/115435 mentions this issue: [release-branch.go1.10] cmd/go: accept more safe CFLAGS/LDFLAGS

Change https://golang.org/cl/115436 mentions this issue: [release-branch.go1.9] cmd/go: accept more safe CFLAGS/LDFLAGS

Hey Guys,
Why is this issue closed when we still cannot build bimg with the latest version of Go as of 9/4/2018 ?
Please see issue: https://github.com/h2non/bimg/issues/230
We cannot build anything that imports bimg since Go 1.10 and the problem still persist in Go 1.11
The error message we get is this:

# go build
go build gopkg.in/h2non/bimg.v1: invalid flag in pkg-config --libs: -Wl,--export-dynamic

Any advise?

EDIT:
I created a new issue: https://github.com/golang/go/issues/27496

I believe this whitelist is responsible for the following: https://github.com/alexflint/gallium/issues/63

@alexflint This issue is closed, please create a new one