Lime: Windows Support [$30]

Wow that sounds fantastic.

martell [email protected] schrieb:

hey @quarnster,

I'm going to take on the task of the windows port.
I should have it done fairly soon.

I have discovered linking errors with mingw64 and go lang and a patch
should be pushed in to the mingw64-crt soon to resolve the last linking
error of pygo and other c binded depends.

---

Reply to this email directly or view it on GitHub:
https://github.com/limetext/lime/issues/228

Thank you!

"C:\msys64\mingw32\pkg\tool\windows_386\8l.exe" -o "C:\msys64\tmp\go-build836004963\lime\3rdparty\libs\gopy_test\gopy.test.exe" -L "C:\msys64\tmp\go-build836004963\lime\3rdparty\libs\gopy_test" -L "C:\msys64\tmp\go-build836004963" "C:\msys64\tmp\go-build836004963\lime\3rdparty\libs\gopy_test\main.a"
$WORK\lime\3rdparty\libs\gopy_test\gopy.test.exe
PASS
ok lime/3rdparty/libs/gopy 2.592s

Great success :)

Wow, that's awesome!

what do I do after gopy? The guide was changed awhile back because cmake was added.
I have also build the qt5 bindings :)

I already have onig build in gcc and obviously python3.3 with .a libs

EDIT: It's okay I'll just follow the steps in the cmakelists manually :)

It would be nice if you tell us how to reproduce the python build.

I'm using msys2 and pacman. mingw-w64-x86_64-python3 is a package for it.
We are usually on irc on OFTC #msys2 added new packages daily :)
You can pick it up here
http://sourceforge.net/projects/msys2/

I recently added go itself but we are working out the issues with it.
When lime is done I'll add it as a package and the PKGBUILD script to build it ;)
python and onig should be here https://github.com/martell/MINGW-packages

I am also on msys2. I played around yesterday with it. How did you generate the static python lib? Did you you use 32bit or 64bit toolchain, go?

Is the mentioned crt fix already online?

https://github.com/martell/MINGW-packages
Python PKGBUILD is in this repo
Yes alexey applied it by hand. Myself and mingwandroid are trying to fix go's CFLAG passing for mingw64
After that it should be good to go :) ... hopefully

Just "pacman -Syu" to update the crt

But python3 is build as a shared lib by default isn't it? With mingw (not w64) they used pexports. How did you handle that? What is wrong with the CFLAGS?

Python3 is compiled with mingw64 so we don't need to do the exports.
We get the libpython3.3m.a by default as a result
https://github.com/martell/MINGW-packages/tree/master/mingw-w64-python3

We don't use visual studio for anything. Nor do we rely on visual studio builds of any library

The equivalent to pexports would be gendef. We don't need it here though
http://sourceforge.net/apps/trac/mingw-w64/wiki/gendef

Okay so I have successfully build the termbox frontend with mingw64.
It closes gracefully with this message in the debug.log
[2014/01/25 00:47:11 GMT] EROR (main.main:607) The handle is invalid.

The qml build gives me this error after setting up the bindings.
Two bindings in the one project must cause issues.
lime/frontend/qml
C:\msys64\tmp\go-build680373563/lime/3rdparty/libs/gopy/lib.a(_all.o): duplicate symbol reference: mingw_onexit in both github.com/niemeyer/qml(.text) and lime/3rdparty/libs/gopy/lib(.text)

https://www.dropbox.com/s/2ibydge1x8k9s1w/lime.7z

I have successfully built the termbox, html and I dropped back in the qt5 main.go as the qml one wouldn't link

Termbox seems the best so far.
I goes through loading all the plugins and then crashes :/
Nice log file though ;)

look in build/bin for exe's and don't move them as they are folder structure relevant

@martell: did you try running the termbox frontend from Windows command prompt instead of mintty?

@EdVanDance
Yes I am running termbox from the windows cmd.prompt
It seems as though with a bit more work on go that this can be resolved.
Also we need to remove setcolormode 256 on windows as we don't have that function

@martell I can't get it to run. I installed i686 python3 (I'm on x86_64). Opened a cmd:

set PYTHONHOME=...\msys64\mingw32\lib\python3.3
termbox.exe

And I get:

Fatal Python error: Py_Initialize: unable to load the file system codec
ImportError: No module named 'encodings'

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

You shouldn't have to set PYTHONHOME as I've included he python dll's
It works for me without that.

Now I maybe wrong but I didn't do that anyways.

The patch has been merged into the mingw64-crt and alexey will be doing an update soon ;)

If I do not set the PYTHONPATH. I get:

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: unable to load the file system codec
ImportError: No module named 'encodings'

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Do you start the command prompt from within msys? I started it directly via Win+R.

No I directly went to cmd like you did, PYTHONHOME is not defined on my system.
I think I previously installed python onto my system with the windows installer,
maybe thats why my cmd windows doesn't complain?

This could explain why it crashes at a certain point also, maybe I'm using the wrong python files

Did you install mingw python3 or msys2 python3 they are very different

Hmm that could be the reason. But I dont't understand why it complains after setting up the PYTHONPATH.
There definetly is a encodings module.

I got it.
It's just:

set PYTHONHOME=...\msys64\mingw32

Without lib\python3.3.

Nice ;) and your output?

Shows the file content, but does not react on key presses. And complains about 256 color mode.

In the zip I uploaded I have all my cgo.go files for reference if you want to build from source ;)
I also have a package on my mingw-packages repo for onig and go which you will need.

Can you pastebin it please :)

I get as far as here

[2014/01/25 04:08:33 GMT] [INFO](lime/backend/sublime.sublime_Console:87) Python sez: Loading plugin Vintageous.jump_list_cmds
[2014/01/25 04:08:33 GMT] [FNST](main.%28tbfe%29.renderthread:375) Rendering
[2014/01/25 04:08:33 GMT] [FNST](lime/backend.%28commandHandler%29.Register:169) Want to register vi_add_to_jump_list
[2014/01/25 04:08:33 GMT] [FNST](lime/backend.%28commandHandler%29.Register:194) Successfully registered command vi_add_to_jump_list
[2014/01/25 04:08:33 GMT] [INFO](lime/backend/sublime.sublime_Console:87) Python sez: Loading plugin Vintageous.motion_cmds
[2014/01/25 04:08:33 GMT] [FNST](main.%28tbfe%29.renderthread:375) Rendering
[2014/01/25 04:08:33 GMT] [FNST](lime/backend.%28*commandHandler%29.Register:169) Want to register irreversible_text

Same here:
http://pastebin.com/G2yNjmdp

Okay Well I'll try and get alexey to merge my go package and onig shared package.
That way everyone can be on the same page build wise.
Here is a guide to build from start to finish reverse engineered from the cmakelists.txt ;)

https://gist.github.com/martell/c178f674188cb779b8fd

Thanks a lot. That definitely helps a lot. Did you also try a 64bit build?

I didn't but it should be the same.
Currently making a script where you can choose which one to build :D

Ok nice. Could you tell me the difference between the msys2 and msys64 base downloads on sourceforge?

I work with Martell and Alexey Pavlov on MSYS2; thought I'd drop by and say "hi".

Some of these details Martell already mentioned but I may as well put it all together.

msys64 was a WIP name, it's just MSYS2 now,.

The only docs (apart from most of Arch Linux's pacman/makepkg stuff) are at:
http://sourceforge.net/p/msys2/wiki/MSYS2%20installation/

There's 3 package repositories, msys, mingw32 and mingw64.

The repository sources are at:
msys: https://github.com/Alexpux/MSYS2-packages
mingw32+mingw64: https://github.com/Alexpux/MINGW-packages

I co-maintain the MinGW-w64 Python packages and patches (of which there are too many unfortunately) so feel free to ask any questions you have and as Martell said, drop by #msys2 on OFTC if you want to chat.

Sorry about not replying. Github never gave me a notification :/
Thanks for clearing that up Ray.
You know more about msys2 and mingw64 than I do so it's better that you explain :)

Ok I see. Thanks for that guys.
For me msys2 is one of the most valuable projects. Especially having pacman is awesome.

Thanks for your kind words. Join us, add packages and share the goodness ;-)

EdVanDance, hi!
MSYS2 has both 32 and 64 bit versions. That's why there are two types of archives to download.

Ok. So msys2 is probably 32bit and msys64 64 bit. :smile:

Edit: Nevermind, there are separate directories for 32 and 64 bit. The recent packages all are named msys2 with a
suffix for 32 bit (i686) and 64 bit (x86_64).

Here is 32-bit MSYS2:
http://sourceforge.net/projects/msys2/files/Base/i686/

And here 64-bit:
http://sourceforge.net/projects/msys2/files/Base/x86_64/

Yeah just realized that. Next time I should have a look before writing not the other way around :smile:

I tend to stick to C:\msys64 and have both mingw32 and mingw64 packages in
there in C:\msys64\mingw32 and C:\msys64\mingw64; pacman handles that for
you though so don't worry about it.

On Tue, Jan 28, 2014 at 4:30 PM, EdVanDance [email protected]:

Yeah just realized that. Next time I should have a look before writing not
the other way around [image: :smile:]

Reply to this email directly or view it on GitHubhttps://github.com/limetext/lime/issues/228#issuecomment-33495293
.

Now that the sources are go get-able on x86/x64 linux/darwin, I'm presuming there might be changes needed here too.

Is there a guide or rough outline of steps needed to get everything installed on a fresh Windows system that doesn't already have go or anything else installed already?

https://github.com/limetext/lime/wiki/Building-on-Windows is certainly in need of a refresh/rewrite...

I will take care of that soon.

Are the Python features required for a working Lime? I tried commenting out import of gopy & sublime (and three lines that depend on it) in both frontend/html and frontend/termbox, and also replaced import of rubex with import (rubex "regexp"); both html & termbox built and run on Windows, but weirdly, in unusable way:

• html had problems interpreting keys (strange symbols like "&" and others appeared both in Firefox and Chromium)
• termbox didn't show cursor, and bottom half of screen was weirdly shifted and rapidly scrolling down on every keypress

any ideas? :)

About the termbox you can get information from log file that is created with every build and you should know we have vintageous plugin that gives some vim features so you should press i first then start typing.

@zoli we weren't able to build for quite awhile as msys updated mingw python to 3.4 which gopy and go 1.3.1 has only just fixed.

Now that the python3.4 issues have been resolved I am looking at using msys2 and mingw-64 and go to build lime text.

Here is my current error

$ go get github.com/limetext/lime/frontend/termbox # github.com/limetext/gopy/lib ..\..\..\mingw64\src\github.com\limetext\gopy\lib\utils.c: In function 'cinit': ..\..\..\mingw64\src\github.com\limetext\gopy\lib\utils.c:503:11: error: 'sigaltstack' undeclared (first use in this function) stub(&sigaltstack); ^ ..\..\..\mingw64\src\github.com\limetext\gopy\lib\utils.c:503:11: note: each undeclared identifier is reported only once for each function it appears in ..\..\..\mingw64\src\github.com\limetext\gopy\lib\utils.c:504:11: error: 'signal' undeclared (first use in this function) stub(&signal); ^ ..\..\..\mingw64\src\github.com\limetext\gopy\lib\utils.c:505:11: error: 'sigaction' undeclared (first use in this function) stub(&sigaction);

It seems this commit breaks gopy on windows
https://github.com/limetext/gopy/commit/8b1545fdde22aad006ba8596dd7e9ad80ff0dec7

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

If I comment them out i get this error
github.com/limetext/gopy/lib(.text): undefined: github.com/limetext/gopy/lib(/4292) github.com/limetext/gopy/lib(.text): undefined: github.com/limetext/gopy/lib(/6573) github.com/limetext/gopy/lib(.text): undefined: github.com/limetext/gopy/lib(/6545) github.com/limetext/gopy/lib(.text): undefined: github.com/limetext/gopy/lib(/6513) too many errors
@quarnster have you any ideas?

here is the build guide for building limetext in msys2 on windows
https://gist.github.com/martell/8588075

a go 1.3.3 package will hit the servers tomorrow hopefully :)

cinit should be ifdef-ed out for windows or something as it's not relevant there. Regarding the linking error, does it find the python lib correctly?

Yes it finds the libraries correctly.
pkg-config is adding the /mingw64/include/python3 include dir.
So I do expect it to also add the lib correctly.
The error message doesn't really help much :(
We are in uncharted territory so i will try to force -lpython3
Post back soon

Okay so here are my results.
pkg-config is finding the lib correctly because here is a test on regex with and without onig installed

without:
$ go test github.com/limetext/rubex testmain github.com/limetext/rubex(.text): undefined: onig_free github.com/limetext/rubex(.text): undefined: onig_number_of_captures github.com/limetext/rubex(.text): undefined: onig_number_of_names github.com/limetext/rubex(.text): undefined: onig_region_free github.com/limetext/rubex(.text): undefined: github.com/limetext/rubex(/63) github.com/limetext/rubex(.text): undefined: onig_region_new github.com/limetext/rubex(.text): undefined: github.com/limetext/rubex(/30) github.com/limetext/rubex(.text): undefined: onig_new github.com/limetext/rubex(.text): undefined: onig_error_code_to_str github.com/limetext/rubex(.text): undefined: onig_search github.com/limetext/rubex(.text): undefined: onig_error_code_to_str github.com/limetext/rubex(.text): undefined: onig_match github.com/limetext/rubex(.text): undefined: onig_name_to_backref_number github.com/limetext/rubex(.text): undefined: onig_foreach_name FAIL github.com/limetext/rubex [build failed]

with:
$ go test github.com/limetext/rubex testmain github.com/limetext/rubex(.text): undefined: github.com/limetext/rubex(/63) github.com/limetext/rubex(.text): undefined: github.com/limetext/rubex(/30) FAIL github.com/limetext/rubex [build failed]

Edit: seems 32bit go on windows prints a more meaningful error

$ go get github.com/limetext/lime/frontend/termbox github.com/limetext/lime/frontend/termbox github.com/limetext/gopy/lib(.text): undefined: _get_output_format github.com/limetext/gopy/lib(.text): undefined: ___lc_codepage_func github.com/limetext/gopy/lib(.text): undefined: ___lc_codepage_func github.com/limetext/gopy/lib(.text): undefined: ___lc_codepage_func github.com/limetext/gopy/lib(.text): undefined: ___lc_codepage_func github.com/limetext/gopy/lib(.text): undefined: ___lc_codepage_func

to solve this link error I must link against the visual studio runtime unfortunately i have to pick a c++ runtime also like v110 or v110 which is annoying for a mingw-w64 application.

okay so now I can actually build :)
Hopefully I will have an exe shortly

EDIT: I can build the termbox but

qml front end gives me this
See same error as above on Jan 25th

$ go build
github.com/limetext/lime/frontend/qml
C:/msys64/mingw32/pkg/windows_386/gopkg.in/qml.v1.a(_all.o): duplicate symbol reference: mingw_onexit in both github.com/limetext/gopy/lib(.text) and gopkg.in/qml%2ev1(.text)

Here is my termbox output:

Seems a bit better than a few months ago but it still crashes

[2014/10/19 11:32:41 BST] [FNST] (github.com/limetext/lime/backend.(*WindowEvent).Call:94) OnNewWindow(2)
[2014/10/19 11:32:41 BST] [FNST] (github.com/limetext/lime/backend.(*ViewEvent).Call:60) OnNew(4)
[2014/10/19 11:32:41 BST] [FNST] (github.com/limetext/lime/backend.(*ViewEvent).Call:60) OnActivated(4)
[2014/10/19 11:32:41 BST] [EROR] (main.setColorMode:614) Unable to use 256 color mode: SetColorMode not implemented on Windows
[2014/10/19 11:32:41 BST] [EROR] (github.com/limetext/lime/backend/packages.func·001:98) Error on walking: GetFileAttributesEx ../../3rdparty/bundles/User: The system cannot find the file specified.
[2014/10/19 11:32:41 BST] [EROR] (github.com/limetext/lime/backend/packages.ScanPackets:110) Can't walk: GetFileAttributesEx ../../3rdparty/bundles/User: The system cannot find the file specified.
[2014/10/19 11:32:41 BST] [INFO] (github.com/limetext/lime/backend.(*Editor).loadKeyBinding:183) Loaded ..\..\packages\Default\Default (OSX).sublime-keymap
[2014/10/19 11:32:41 BST] [FNST] (github.com/limetext/lime/backend.(*Editor).Watch:414) Watch(&{0x12760400})
[2014/10/19 11:32:41 BST] [INFO] (github.com/limetext/lime/backend.(*Editor).loadKeyBinding:183) Loaded ..\..\packages\Default\Default.sublime-keymap
[2014/10/19 11:32:41 BST] [FNST] (github.com/limetext/lime/backend.(*Editor).Watch:414) Watch(&{0x12760420})
[2014/10/19 11:32:41 BST] [INFO] (github.com/limetext/lime/backend.(*Editor).loadKeyBinding:183) Loaded ..\..\packages\Default\Default (OSX).sublime-keymap
[2014/10/19 11:32:41 BST] [FNST] (github.com/limetext/lime/backend.(*Editor).Watch:414) Watch(&{0x127ce180})
[2014/10/19 11:32:41 BST] [INFO] (github.com/limetext/lime/backend.(*Editor).loadKeyBinding:183) Loaded ..\..\packages\Default\Default.sublime-keymap
[2014/10/19 11:32:41 BST] [FNST] (github.com/limetext/lime/backend.(*Editor).Watch:414) Watch(&{0x127ce1a0})
[2014/10/19 11:32:41 BST] [INFO] (github.com/limetext/lime/backend.(*Editor).loadSetting:199) Loaded ../../packages/Default/Preferences.sublime-settings
[2014/10/19 11:32:41 BST] [FNST] (github.com/limetext/lime/backend.(*Editor).Watch:414) Watch(&{0x127e83b0})
[2014/10/19 11:32:41 BST] [INFO] (github.com/limetext/lime/backend.(*Editor).loadSetting:199) Loaded ../../packages/Default/Preferences (Windows).sublime-settings
[2014/10/19 11:32:41 BST] [FNST] (github.com/limetext/lime/backend.(*Editor).Watch:414) Watch(&{0x127d2270})
[2014/10/19 11:32:41 BST] [EROR] (github.com/limetext/lime/backend/packages.(*Packet).Get:47) Couldn't read file: open ../../3rdparty/bundles/User/Preferences.sublime-settings: The system cannot find the path specified.
[2014/10/19 11:32:41 BST] [EROR] (github.com/limetext/lime/backend.(*Editor).loadSetting:197) 1,1: Unexpected EOF, 0-0: "JSON" - Data: ""

[2014/10/19 11:32:41 BST] [FNST] (main.(*tbfe).renderthread:437) Rendering
[2014/10/19 11:32:41 BST] [FNST] (main.(*tbfe).renderthread:437) Rendering
[2014/10/19 11:32:41 BST] [FNST] (main.(*tbfe).renderthread:437) Rendering

Nice work! What's the backtrace for the crash?

If you launch it with "./termbox main.go" does it try and display this file correctly?

If you comment out the gopy import + calls, does it still crash? If not, it might be one of the standard Go mingw issues.

@quarnster: I have no idea about the back trace :(
I don't understand go lang well enough.
How do I even debug it ?

I don't get to see the output here is the log though

[2014/10/19 15:26:09 BST] [FNST] (github.com/limetext/lime/backend.(*WindowEvent).Call:94) OnNewWindow(2)
[2014/10/19 15:26:09 BST] [FNST] (github.com/limetext/lime/backend.(*ViewEvent).Call:60) OnNew(4)
[2014/10/19 15:26:09 BST] [FNST] (github.com/limetext/lime/backend.(*ViewEvent).Call:60) OnActivated(4)
[2014/10/19 15:26:09 BST] [FNST] (github.com/limetext/lime/backend.(*ViewEvent).Call:60) OnModified(4)
[2014/10/19 15:26:09 BST] [FNST] (github.com/limetext/lime/backend.(*ViewEvent).Call:60) OnSelectionModified(4)
[2014/10/19 15:26:09 BST] [FNST] (github.com/limetext/lime/backend.(*ViewEvent).Call:60) OnLoad(4)
[2014/10/19 15:26:09 BST] [FNST] (github.com/limetext/lime/backend.(*Editor).Watch:414) Watch(C:\msys64\mingw32\src\github.com\limetext\lime\frontend\termbox\main.go (4))
[2014/10/19 15:26:09 BST] [FNST] (github.com/limetext/lime/backend.(*ViewEvent).Call:60) OnDeactivated(4)
[2014/10/19 15:26:09 BST] [FNST] (github.com/limetext/lime/backend.(*ViewEvent).Call:60) OnActivated(4)

I will try removing the gopy imports and see

@martell, Backtrace should be dumped to the terminal normally, except in this case we are hijacking the terminal to render some UI so it won't work ;)

Try this patch and see if you get something useful in the log:

diff --git a/frontend/termbox/main.go b/frontend/termbox/main.go
index 4ff2426..bdecd51 100644
--- a/frontend/termbox/main.go
+++ b/frontend/termbox/main.go
@@ -711,7 +711,9 @@ func main() {
                termbox.Close()
                log4go.Debug(util.Prof)
                if err := recover(); err != nil {
-                       log4go.Crash(err)
+                       log4go.Debug(err)
+                       log4go.Debug("%s", string(debug.Stack()))
+                       time.Sleep(time.Second) // TODO: see issue #260
                }
        }()

Can this thread be reset into new issue with status update?

@techtonik No need to close :)
I will try updating the packages we have for go and do a few patches on them to see where we stand on this now

Okay so it seems that currently we are still at the same stage where part of the go language itself does not work properly on windows.

I am exploring 2 different possible solutions to this.
Both will require a bit of time to test.
I'll be back in about a week with an update

Week is passed. This issue needs to be reticketed with short summary - for now it is impossible to take action on it given all the limitation of real life and free time.

martell, which part of the go language fails to work properly on windows? Also I think you mean the go implementation rather than the language itself?

Hi! What news about porting on windows? Do you have any progress?

I don't entirely know what the problems used to be, but Lime-QML builds and runs fine for me in Windows. Here's my instructions: https://github.com/ricochet1k/lime-qml/wiki/Building-on-Windows-using-MSYS2

The year is 2017, the month is June, i.e. 1250 days passed since issue was created.
Err… Hello? Could we try some pre-compiled Windows binary at last, please?

have discovered linking errors with mingw64 and go lang and a patch
should be pushed in to the mingw64-crt soon to resolve the last linking
error of pygo and other c binded depends

unsub me from this

Here..