Libelektra: How to get the typechecker plugin into kdb

As far as I know there is no typechecker binding. You need the typechecker plugin, which requires the haskell binding. For an example, on how to get the thing running on macOS (the steps on Linux should be pretty much the same), please take a look at the 🍏 Haskell build job in .travis.yml.

Yes. you need the haskell binding (It has quite some deps: https://www.libelektra.org/bindings/haskell), then the haskell plugin (again deps: https://www.libelektra.org/plugins/haskell) so that you can finally enable the typechecker plugin.

@e1528532 Can you update the docu of the typechecker which parts are needed so that it will work? And why is infos/needs in the contract duplicated?

This installation process is extremely cumbersome, especially for linux. You require GHC (Glasgow Haskell Compiler) > 8.0.1 && < 8.4. The package manager just has 7.x, stack already gives you 8.4.3.
After tagging a specific version I still receive an error...

1) On various places i get the bug:

CMake Error at cmake/Modules/FindHaskell.cmake:61 (string):
  string sub-command REPLACE requires at least four arguments.
Call Stack (most recent call first):
  src/plugins/typechecker/CMakeLists.txt:7 (find_package)

2)
-- Exclude Binding haskell because ghc: found, but 8.0.1 required
When I run ghc --version I receive the The Glorious Glasgow Haskell Compilation System, version 8.2.2

3)

-- Exclude Plugin typechecker because GHC_RTS_LIB not found

What is this? Where can i find it?

My cmake version: cmake version 3.5.1

What linux do you use that still only has 7.x? Is it possible that you have multiple versions of ghc installed and that cmake picks the wrong one? The first bug you describe is probably really a bug, i guess older ghc versions don't output something for the --print-target-platform command line parameter and thus replace complains. will fix. The second issue also indicates that it probably picks some old ghc version that doesn't support the keywords i need, even not those to fetch its version number. What happens if you add something like message "Used ghc at ${GHC_EXECUTABLE}" to FindHaskell.cmake and check what ghc it exactly uses? Is it really 8.2.2 (i use that on macOS as well)?

I agree that having < 8.4 unsupported is annoying but newer versions of a libraries used are not supported on 8.0.1 due to https://github.com/mvdan/hint/pull/63 so until this is fixed i can't support newer versions unfortunately.

Linux Mint 18.2 which uses the xenial ubuntu repository.

Is it possible that you have multiple versions of ghc installed and that cmake picks the wrong one?

Yes, it does. It tries to use opt/ghc/7.10.3/bin/ghc which was installed via the package manager.
After purging it, it is still used though by cmake. I installed the newer version via stack and set the PATH in my zshrc to use the correct one.

ok i will try to improve the cmake file to detect versions 7.x and give appropriate hints in that case. As far as i know cmake caches the programs it found though so that could be a reason why it resumed to use 7.x.

After cleaning the build directory it works.

But now when calling make it fails:

Scanning dependencies of target haskell
[ 21%] Generating libHShaskell-ghc8.2.2.so
cabal: Error: some packages failed to install:
libelektra-haskell-0.8.23-LGvezP3EsDCKBrt9zZKeXV failed during the final
install step. The exception was:
ExitFailure 1

setup: Encountered missing dependencies:
libelektra-haskell ==0.8.23
src/plugins/haskell/CMakeFiles/haskell.dir/build.make:64: recipe for target 'src/plugins/haskell/libHShaskell-ghc8.2.2.so' failed
make[2]: *** [src/plugins/haskell/libHShaskell-ghc8.2.2.so] Error 1
CMakeFiles/Makefile2:4033: recipe for target 'src/plugins/haskell/CMakeFiles/haskell.dir/all' failed
make[1]: *** [src/plugins/haskell/CMakeFiles/haskell.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

I assume you have installed the sandbox with the dependencies? what happens if you cd to build/src/bindings/haskell and call cabal build, or if that fails, cabal install --only-dependencies?

I assume you have installed the sandbox with the dependencies?

Yes

it doesnt fail. it writes

Preprocessing library for libelektra-haskell-0.8.23..
Building library for libelektra-haskell-0.8.23..

Ok, i called cabal install myself now in build/src/bindings/haskell and now it works and builds the whole project. I don't know why it is working though

PS: I needed to do the same in src/plugins/typechecker

Alright ... here is your next problem ...

Scanning dependencies of target elektra-type
[ 44%] Building CXX object src/plugins/type/CMakeFiles/elektra-type.dir/type.cpp.o
[ 44%] Linking CXX shared module ../../../lib/libelektra-type.so
[ 44%] Built target elektra-type
[ 44%] Built target typechecker-ghc-plugin
[ 44%] Built target typechecker
[ 44%] Building C object src/plugins/typechecker/CMakeFiles/elektra-typechecker-objects.dir/haskell.c.o
/media/wespe/extended/repository/piankero-libelektra/build/src/plugins/typechecker/haskell.c:12:30: fatal error: Typechecker_stub.h: No such file or directory
compilation terminated.
src/plugins/typechecker/CMakeFiles/elektra-typechecker-objects.dir/build.make:62: recipe for target 'src/plugins/typechecker/CMakeFiles/elektra-typechecker-objects.dir/haskell.c.o' failed
make[2]: *** [src/plugins/typechecker/CMakeFiles/elektra-typechecker-objects.dir/haskell.c.o] Error 1
CMakeFiles/Makefile2:8408: recipe for target 'src/plugins/typechecker/CMakeFiles/elektra-typechecker-objects.dir/all' failed
make[1]: *** [src/plugins/typechecker/CMakeFiles/elektra-typechecker-objects.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

Ok so the bindings seem to build correctly. Next step in debugging this: Whats the output of cabal install --only-dependencies in the folder build/srcu/plugins/haskell/? And what does cabal sandbox hc-pkg list in the same folder show? Seems to be some kind of dependency issue and i am unsure why it happens, maybe because of the initial errors.

Some add-source dependencies have been modified. They will be reinstalled...
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] next goal: typechecker (user goal)
[__0] rejecting: typechecker-1.0.8.23/installed-9qb... (package is broken)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: typechecker
Note: when using a sandbox, all packages are required to have consistent
dependencies. Try reinstalling/unregistering the offending packages or
recreating the sandbox.

That file should be generated in build/src/plugins/typechecker/dist/build/Elektra. Whats in that folder? To me it seems as if the initial errors messed up cmake somehow and left the build in a broken state. Unfortunately it is quite hard to marry cmake and cabal thus the installation may be a bit cumbersome.

A solution that doesn't require redoing everything would be deleting the haskell-related things from the build directory, so rm -rf build/src/libs/typesystem && rm -rf build/src/plugins/haskell && rm -rf build/src/plugins/typechecker && rm -rf build/src/plugins/regexdispatcher && rm -rf build/src/bindings/haskell, then calling cmake . again and compile.

And what does cabal sandbox hc-pkg list in the same folder show?

/home/wespe/.stack/programs/x86_64-linux/ghc-8.2.2/lib/ghc-8.2.2/package.conf.d
    Cabal-2.0.1.0
    array-0.5.2.0
    base-4.10.1.0
    binary-0.8.5.1
    bytestring-0.10.8.2
    containers-0.5.10.2
    deepseq-1.4.3.0
    directory-1.3.0.2
    filepath-1.4.1.2
    ghc-8.2.2
    ghc-boot-8.2.2
    ghc-boot-th-8.2.2
    ghc-compact-0.1.0.0
    ghc-prim-0.5.1.1
    ghci-8.2.2
    haskeline-0.7.4.0
    hoopl-3.10.2.2
    hpc-0.6.0.3
    integer-gmp-1.0.1.0
    pretty-1.1.3.3
    process-1.6.1.0
    rts-1.0
    template-haskell-2.12.0.0
    terminfo-0.4.1.0
    time-1.8.0.2
    transformers-0.5.2.0
    unix-2.7.2.2
    xhtml-3000.2.2
/media/wespe/extended/misc/cabal_sandbox/.cabal-sandbox/x86_64-linux-ghc-8.2.2-packages.conf.d
    HUnit-1.6.0.0
    QuickCheck-2.11.3
    ansi-terminal-0.8.0.4
    auto-update-0.1.4
    call-stack-0.1.0
    case-insensitive-1.2.0.11
    clock-0.7.2
    colour-2.3.4
    cpphs-1.20.8
    easy-file-0.2.2
    exceptions-0.8.3
    fast-logger-2.4.11
    ghc-paths-0.1.0.9
    hashable-1.2.7.0
    haskell-src-exts-1.20.2
    hint-0.7.0
    hspec-2.5.5
    hspec-core-2.5.5
    hspec-discover-2.5.5
    hspec-expectations-0.8.2
    integer-logarithms-1.0.2.1
    libelektra-haskell-0.8.23
    libfa-1.0.8.23
    megaparsec-6.5.0
    mtl-2.2.2
    old-locale-1.0.0.7
    old-time-1.1.0.3
    parser-combinators-1.0.0
    polyparse-1.12
    primitive-0.6.4.0
    quickcheck-io-0.2.0
    random-1.1
    scientific-0.3.6.2
    setenv-0.1.1.3
    simple-logger-0.0.4
    specelektra-1.0.8.23
    spectranslator-1.0.8.23
    stm-2.4.5.0
    temporary-1.3
    text-1.2.3.0
    tf-random-0.5
    transformers-compat-0.6.2
    typechecker-1.0.8.23
    unix-time-0.3.8

That file should be generated in build/src/plugins/typechecker/dist/build/Elektra. Whats in that folder?

Typechecker.dyn_hi Typechecker.dyn_o Typechecker.hi Typechecker.o Typechecker_stub.h

Still receiving the error

[ 44%] Generating libHStypechecker-ghc8.2.2.so
In order, the following would be installed:
libelektra-haskell-0.8.23 (via: spectranslator-1.0.8.23) (reinstall)
libfa-1.0.8.23 (via: spectranslator-1.0.8.23 specelektra-1.0.8.23) (reinstall)
specelektra-1.0.8.23 (via: spectranslator-1.0.8.23) (reinstall)
spectranslator-1.0.8.23 (reinstall)
cabal: The following packages are likely to be broken by the reinstalls:
typechecker-1.0.8.23
Use --force-reinstalls if you want to install anyway.

cabal: Could not resolve dependencies:
[__0] next goal: typechecker (user goal)
[__0] rejecting: typechecker-1.0.8.23/installed-9qb... (package is broken)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: typecheckerNote: when using a sandbox,
all packages are required to have consistent dependencies. Try
reinstalling/unregistering the offending packages or recreating the sandbox.

Can we maybe reproduce this problem in a docker image? Would be also nice to get some more distros into our build server, seems like there are some important differences after all, at least in Haskell.

@e1528532 any progress here? Actually we even have a Docker image for xenial: scripts/docker/ubuntu/xenial/Dockerfile so you would just need to add the Haskell stuff.

@Piankero have you tried my suggestion with deleting the haskell things in the build folder any calling cmake again? Did that help?

@markus2330 i can try to setup haskell on xenial yes, but i'd need to use a different version of ghc if xenial only delivers 7.x. I need to use 8.0 or higher because 7.x lacks the required extension, is unsupported by a number of libraries like hint also, and apparently lacks a number of command line things i use in cmake.

@e1528532 using the same compiler as @Piankero used.

@Piankero can you be more specific which versions you use, what commands you typed and so on.

have you tried my suggestion with deleting the haskell things in the build folder any calling cmake again? Did that help?

No, it did not help. I even removed the whole build folder and tried it again without success.

Scanning dependencies of target haskell
[ 20%] Generating libHShaskell-ghc8.2.2.so
cabal: Could not resolve dependencies:
[__0] trying: haskell-1.0.8.24 (user goal)
[__1] trying: libelektra-haskell-0.8.24 (user goal)
[__2] next goal: typechecker (user goal)
[__2] rejecting: typechecker-1.0.8.23/installed-9qb... (conflict:
libelektra-haskell==0.8.24, typechecker =>
libelektra-haskell==0.8.23/installed-LGv...)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: libelektra-haskell, haskell,
typechecker
Note: when using a sandbox, all packages are required to have consistent
dependencies. Try reinstalling/unregistering the offending packages or
recreating the sandbox.

setup: Encountered missing dependencies:
libelektra-haskell ==0.8.24
src/plugins/haskell/CMakeFiles/haskell.dir/build.make:64: recipe for target 'src/plugins/haskell/libHShaskell-ghc8.2.2.so' failed
make[2]: *** [src/plugins/haskell/libHShaskell-ghc8.2.2.so] Error 1
CMakeFiles/Makefile2:4033: recipe for target 'src/plugins/haskell/CMakeFiles/haskell.dir/all' failed
make[1]: *** [src/plugins/haskell/CMakeFiles/haskell.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

can you be more specific which versions you use, what commands you typed and so on.

mkdir build && cd build
ccmake ..
#Added `;haskell` to BINDING
#Added `;haskell;typechecker;` to PLUGIN
cmake ..
make

which versions you use

Still not answered. The version of cabal, ... might be interesting. And where did you get GHC from? Seems like Xenial only has GHC 7? Reproducing in a docker image seems like best way forward. There are many things that might be broken locally (e.g. HASKELL_SHARED_SANDBOX could be set and contain garbage?)

Right this is also possible. If you created the HASKELL_SHARED_SANDBOX with ghc7 beforehand i can imagine that there are some incompatibilities with ghc8 then. As far as i've understood you manually installed ghc8.2.2 then in xenial? If so, i will try to setup a docker image that uses that version and build the project on our xenial image as well.

Also, the cabal version used is important i think, not sure if the versions shipped with ghc7 work well, i assumed 8.0.1 as present on stretch as the minimum that corresponds to cabal 1.24.1.0 as stated in the haskell binding dependencies.

what are the versions of alex, c2hs and happy? If you install them via cabal, ensure that ghc8 is used (and an appropriate cabal version)

Interesting, tried your setup in a docker container (xenial with ghc7.10) and when i configure the project i get:
-- Exclude Binding haskell because ghc: 7.10.3 found, but 8.0.1 required
-- Exclude Plugin haskell because GHC_BASE_LIB not found

i can't check for exclusion of the bindings in a plugin in a good way. It would be great if i can have plugins depend on bindings, but i don't like working with cmake ;) as a workaround i've added the version check to LibAddHaskellPlugin as well.

CMake Error at cmake/Modules/FindHaskell.cmake:61 (string): string sub-command REPLACE requires at least four arguments.

Can't reproduce this unfortunately, but i've added a (blind) workaround to the xenial branch

Versions:

• cabal: 2.2.0.0
• alex: 3.1.6
• c2hs: 0.27.1 Eternal Sunshine, 29 November 2015
  build platform is "x86_64-linux" <1, True, True, 1>
• happy: 1.19.5

If you install them via cabal, ensure that ghc8 is used

I did not install via cabal as your tutorial states it should be installed via the package manager.

I downloaded ghc8.2 via stack and set my PATH in zshrc to the binary. The same for the HASKELL_SHARED_SANDBOX.

Ok thanks, apparently these are also older versions than i've tested it with. So it could be that for instance the older c2hs version doesn't correctly compile the bindings, leading to the missing dependency error you encounter.
I think its probably better to install them into the sandbox in appropriate versions instead to avoid having outdated OS packages. The sandbox is used to be as independent as possible from anything that is installed in target systems. The cabal version seems good though.

Lets look what happens in the PR that adds ghc to the xenial docker image.

I recreated the whole sandbox, installed happy/c2hs/alex via cabal to have more recent versions and manually set the path to the cabal sandbox bin directory.

I still receive errors.
First it complained

setup: Encountered missing dependencies:
libelektra-haskell ==0.8.24

src/plugins/haskell/CMakeFiles/haskell.dir/build.make:64: recipe for target 'src/plugins/haskell/libHShaskell-ghc8.2.2.so' failed
make[2]: *** [src/plugins/haskell/libHShaskell-ghc8.2.2.so] Error 1
CMakeFiles/Makefile2:4033: recipe for target 'src/plugins/haskell/CMakeFiles/haskell.dir/all' failed
make[1]: *** [src/plugins/haskell/CMakeFiles/haskell.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

I manually went into the directory build/src/plugin/haskell and called cabal install there. Then I had that dependency at least.

After cleaning the build directory again I am now stuck here:

Scanning dependencies of target c2hs_haskell
[ 20%] Generating libHSlibelektra-haskell-ghc8.2.2.so
cabal: Could not resolve dependencies:
[__0] next goal: haskell (user goal)
[__0] rejecting: haskell-1.0.8.24/installed-HvL... (package is broken)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: haskellNote: when using a sandbox, all
packages are required to have consistent dependencies. Try
reinstalling/unregistering the offending packages or recreating the sandbox.

Saved package config file is outdated:
• the Cabal version changed from Cabal-2.0.1.0 to Cabal-2.2.0.1
Re-run the 'configure' command.
src/bindings/haskell/CMakeFiles/c2hs_haskell.dir/build.make:75: recipe for target 'src/bindings/haskell/libHSlibelektra-haskell-ghc8.2.2.so' failed
make[2]: *** [src/bindings/haskell/libHSlibelektra-haskell-ghc8.2.2.so] Error 1
CMakeFiles/Makefile2:12336: recipe for target 'src/bindings/haskell/CMakeFiles/c2hs_haskell.dir/all' failed
make[1]: *** [src/bindings/haskell/CMakeFiles/c2hs_haskell.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

This typechecker plugin is really getting annoying ...

i feel you, getting this whole compilation going with cabal and cmake was/is a pain. i still consider switching to stack instead of cabal, though it implies an extra tool i think it could help making this more stable.

i really wonder whats the big difference in your setup. the xenial docker image succeeds to compile everything so it shouldn't be impossible with that setup. But for now i suggest that you leave it be i try to find a good solution.