Laverna: Sync to non-commercial storage

Or Owncloud...

Look at vole.cc. They have a custom Go-based server that saves stuff to files. You could probably just plug in into this for easy local storage.

Yes plug onto an owncloud will be perfect

Syncing to OwnCloud would be interesting.

A Bittorrent Sync option would also be nice.

+1 for owncloud

As @michielbdejong wrote about StackEdit, inside there one might find interesting approaches to a generalized multi-provider data storage connector.

Also, remoteStorage is now implemented, too.

+1 for owncloud !

+1

+1 Cozy.io, Owncloud, Bittorrent Sync. Any one of those would be great!

I think this thread demonstrates the need for a standardized method that connects Laverna to storage providers. Everyone has their own preferred program for hosting personal data. There's nothing wrong with that, but now that the developers have provided viable commercial (Dropbox) and non-commercial (RemoteStorage) means of doing so, I would prefer they focus on expanding other aspects of the project. Giving users the tools to connect on their own would free up time in the long run.

I would also like Google Drive syncing (or even Google Keep sync)

+1 - I also agree with @andtheWings.

But for now I think adding 2 services: GoogleDrive (primary, since most people are already using it) and OwnCloud (extensively used Dropbox non-commercial alternative) would cover a very large user base and open Laverna notes to many more people.

This can really expand the community.

Please note remoteStorage brings Google Drive and Dropbox sync out-of-the-box. Check at 5apps.com.

_Edit:_ Bittorrent Sync in the browser sounds adventurous.

@almereyda Err, could you be more specific how can I store my notes through remoteData to google drive with 5apps.com?

@almereyda If by 'out-of-the-box syncing', you mean in your own control, then so does OwnCloud.

But my point is to spread Laverna usage and make it easily adopted by the general public. I believe Google Drive is the missing link there. Most people have a Google account and they can directly use their Google Drive vs restarting with another service signup and setup.

Since most people here prefer OwnCloud (including me) and it is open/non-commercial, I would definitely recommend an option for that being developed first over Google Drive.

I chose to use Laverna, hoping it might evolve into something able to replace Evernone. I'm using OwnCloud to replace Dropbox, Google Drive ect. I took these steps so that I would no longer have to depend on those services and agree with their terms. I get that adding the option to sync with services like Dropbox would add to the popularity (and I encourage that) , but for me personally it doesn't fit.

@SandeepPinge

Hurray !

GDrive for the masses / Owncloud for the real ones !
imho, Owncloud / GDrive makes an excellent free / proprietary software combo.

I'm running Laverna on my own webspace. Why I can't save the data (encrypted) directly to the webserver? Or can I implement RemoteStorage on my webserver? I haven't SSH access!

I think that Laverna was not thought to be a hosted application, but runs local with locally storing the files. I would prefer a hosted solution as well, since it makes synchronisation between devices much easier.

Local?
If I want local notes I use notepad.exe.
;-)

Laverna was designed to be unhosted, i.e. for the data to be stored locally on the browser by default. More info here: https://unhosted.org/

"None of us can get access to your personal data because we are using IndexedDB and localStorage. In fact all your information will be stored only on client-side."
This means, afaik that the data is stored locally (in a folder that can be accessed by the browser). But one can use Dropbox (and hopefully other alternatives soon) to sync the notes.

Everyone, please note that remoteStorage is now available in Cozy Cloud, so you can easily spin your own instance.

Additionally, as Laverna supports remoteStorage, but it seems an old spec, already, there is work going on to integrate it into ownCloud, too.

@michielbdejong @skddc Could you have a quick look at Laverna and inspect why it fails to sync with 5apps? It used to work, I believe before the 0.10 release. Did they just not update the client? Would it make sense to submodule it?

I'm not familiar with Marionette or this app's source code. However, we're always here to help and answer and all questions, both general as well as for concrete code.

Whoever maintains RS support in this app (or maintains the app in general): we're very much looking forward to welcoming you in #remotestorage on Freenode or chat about anything on the forums or GitHub.

I would add Syncthing (https://syncthing.net/) and Seafile (http://seafile.com/en/home/) as well.

1+ for OwnCloud!

1+ OwnCloud

Owncloud would be great.

Owncloud +1

owncloud +1

owncloud +1
I'm in excactly the same situation as @Putdeksel
Being completely in control of your notes by using your own storage system would be amazing.

Being completely in control of your notes by using your own storage system would be amazing.

Yes, it _is_ amazing and it's already possible via Laverna's RemoteStorage support. You're not even tied to a single server implementation, but any server speaking the RS protocol can store your notes. ;)

@skddc I haven't looked into RemoteStorage to be honest. But sadly from what i can see it doesn't seem to be supported on the server i have from my provider. Hopefully it will in the nearest future!

+1 ownCloud.

OwnCloud sync would be great.

+1 for owncloud!

May I suggest something to all the ownCloud users in here:

It would be _much_ easier for app developers (not just of this app) to support ownCloud, if ownCloud were to support an open protocol for per-user data storage. This app already supports remoteStorage, so if ownCloud were to support remoteStorage, instead of developers having to add special ownCloud support to their apps, then all remoteStorage-enabled apps would automatically also work with ownCloud acting as the remote storage server.

Hence, I think it'd be great if you would ask/vote for, or contribute RS support to ownCloud itself!

Following this conversation for a few years already, I have to second
@skddc 's latest remark.

On 25 July 2016 at 14:22, Sebastian Kippe [email protected] wrote:

May I suggest something to all the ownCloud users in here:

It would be _much_ easier for app developers (not just of this app) to
support ownCloud, if ownCloud were to support an open protocol for per-user
data storage. This app already supports remoteStorage, so if ownCloud were
to support remoteStorage, instead of developers having to add special
ownCloud support to their apps, then all remoteStorage-enabled apps would
automatically also work with ownCloud acting as the remote storage server.

Hence, I think it'd be great if you would ask/vote for, or contribute RS
support to ownCloud itself!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Laverna/laverna/issues/6#issuecomment-234938265, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABka_MqpAc9pic432ehmtZ58HBUTh2Iyks5qZKqOgaJpZM4BqQ_m
.

I +1 this issue!

Hmm would it be hard to implement webDAV as this could be seen as an open standard which will allow laverna to be connected to dozens of different storage providers?

Why not store the data in a specified local directory and people can sync their data into cloud via sync clients like Seafile or ownCloud Desktop clients?

Why not store the data in a specified local directory and people can sync their data into cloud via sync clients like Seafile or ownCloud Desktop clients?

That's only possible in a potential packaged version of the app, but not on the Web. It's also what remoteStorage solves and why it's used in Laverna.

Did anyone here open an ownCloud issue about integrating remoteStorage so far? I can only stress how great that would be and that Laverna would then automatically support ownCloud.

That's only possible in a potential packaged version of the app, but not on the Web. It's also what remoteStorage solves and why it's used in Laverna.

I actually never succeeded to have a fully functional remoteStorage synchronization . I tried many times, and it's rather hard to deal with everybody everytime. I am not so comfortable with IndexedDB datas...

I am rather familiar with the DAVs syncs. @wwebfor told me there's another way to sync in the oven, thought.

I actually never succeeded to have a fully functional remoteStorage synchronization . I tried many times, and it's rather hard to deal with everybody everytime.

Are you saying it failed after the sync fixes a while back, and you opened an issue about it, that didn't come to a conclusion?

I haven't seen any such issues or requests, but I'd be happy to help you! Not sure what "hard to deal with everybody everytime" means.

I am not so comfortable with IndexedDB datas

Well, it's basically the only database available locally in modern browsers (except for localStorage of course), so there aren't really options for using something else.

Are you saying it failed after the sync fixes a while back, and you opened an issue about it, that didn't come to a conclusion?

No, I asked on Gitter, and I came to the conclusion that I must clean all my data on every browser I used as it appears that it properly works on clean browser (where Laverna have never been used).

Not sure what "hard to deal with everybody everytime" means.

Each browser (or desktop client) I use, each time i use one of them.

Well, it's basically the only database available locally in modern browsers (except for localStorage of course), so there aren't really options for using something else.

Yes, yes, I understand. But I meant that I am not used to this way of syncing datas.

Sounds like some upgrade issue to me. So, when you write...

I actually never succeeded to have a fully functional remoteStorage synchronization

... does that mean it still doesn't work? If that's the case, then I'd still be happy to help.

+1 for webDAV, much more standard than an "Owncloud only" solution. Let's stay Open guys.

Instead of integrating a whole lot of nonsensical new hipster solutions like remoteStorage, why not use tried-and-tested webDav? I mean what's the point? Why are you forcing users to bend to your will if industry standard is de facto something else?

Saying yeah, rs is supporting this and that is of little use to most of the folks. But I do not want to use yet another protocol I do not care about. You support commercial Dropbox, but OWNING the notes means I do not want to save them to a 3rd party server. You just do not want to understand your software will survive only if people want to use it because its friendliness and flexibility. Instead, you aim to strong-arm them into something they don't want to do. These are all devs and tech-savvy people, you just don't get this simple fact.

@technodrome I'm not speaking for the Laverna devs, but as a remoteStorage.js developer. Maybe it's not clear, that my trying to help people here (I never tried anything else, if you carefully read the history) is not the same as telling people to go away, because their opinions aren't valid or heard by someone else. This is certainly the right place to voice them as far as I can see, and I'm not sure why your first comment in this thread is so hostile. Let's all be nice to each other, because this is unpaid voluntary work for all of us!

Instead, you aim to strong-arm them into something they don't want to do

I don't see how providing an open-source software, free of charge, is strong-arming anyone. If these are all devs and tech-savvy people, surely someone can contribute WebDAV support, if it makes sense?

So, let's look at the viability of WebDAV objectively then. Here is a (probably incomplete) list of the problems I can think of, that would need to be solved for WebDAV to be integrated. (These are basically the very reasons why the remoteStorage protocol was created in the first place. In fact, RS was using WebDAV in the very beginning, and the authors decided to remove it in favor of a simpler REST API, based on real-world testing and experience with that):

1. CORS

The way browsers work is that an unhosted app like Laverna connects directly to the server from JavaScript. This carries with it the restriction, that the server must offer Cross Origin Resource headers for all requests and support OPTIONS pre-flight requests for HTTP verbs that can manipulate data, like e.g. POST, PUT, DELETE. Most WebDAV servers do not support this. So a lot of WebDAV servers would be incompatible with Laverna out of the box.

Problem: even if one would not care about those servers being incompatible, how would an app detect if support is there? This is actually made much more difficult by browsers intentionally not giving JS code details about why the connection failed, so there would need to be some kind of discovery mechanism. Also, it has to be explained to the user what exactly could be the cause and that they might have to switch servers in order to use the app.

Solution in RS: the protocol requires CORS out of the box. Also, discovery is handled via Webfinger, where the user address can return all configuration info, but also where extra capabilities of the server can be announced.

Solution for plain WebDAV/Laverna: Any ideas?

2. Permissions / Authorization

Permissions have to be handled server-side and per account in WebDAV, as there's no such thing built into the protocol. Authorization of certain folders to access doesn't exist.

Solution in RS: OAuth 2.0, so app developers can decide what part of the storage they need access to, and users can easily see which part that is, and decide to grant access in a simple dialog.

Solution for plain WebDAV/Laverna: This is basically impossible, but it would of course be possible to just skip that feature and voluntarily give Laverna access to all of one's data. Not elegant, but certainly not a showstopper for implementation.

3. Offline/mobile support

This is not particularly a problem of WebDAV or solved in the RS specification alone entirely. But for a mobile-capable web app, it is nonetheless an important factor. Both desktop and mobile connections drop regularly, and one wouldn't want their notes to not be saved because of it. So some kind of offline storage needs to be used, and intelligently synced with the remote server (and of course without re-auth after every drop if you want to create a good user experience).

There are two problems here:

Problem 1: Servers need to support Etags, so one is able to build a sync mechanism in the first place

Solution in RS: This is partly solved in the protocol by requiring ETags on both directory resources and their item listings, as well as single documents themselves. Based on that, one can build sync mechanisms using HTTP headers like e.g. If-None-Match and responses like e.g. empty 304s for requesting things that haven't changed and 412 for refusing to overwrite something that has changed on a different device at a different time.

Solution for plain WebDAV/Laverna: Unfortunately, Etag support is only a SHOULD in the WebDAV spec, so many servers don't support it, and some even ask their users explicitly to turn off support, even if they have it, because it's implemented in a non-performant way. (This is actually not an easy problem for a server developer; I can certainly attest to that). But similar to CORS, it is possible, so the solution here would again be down to feature discovery and guiding users to configure their server or tell them they have to switch to another server. Fortunately, this is much easier than with CORS, because one can detect this from JS by inspecting response headers.

Problem 2: Write an actual sync mechanism or use a library that provides one

Solution in RS: Very early on, one of the spec authors started creating a reference client library to solve this issue for people who want to integrate RS into their apps. It is still being maintained and developed and has been battle-tested on hundreds of thousand of different connections, devices, OSs, browsers, in Cordova, and so on. It is also being used in Laverna. The first commit of that goes back to Nov 2010, so that certainly isn't a "new hipster solution". If there are bugs with it, there are people who are glad to help with those, and any feedback was and is always appreciated (hence my proactive responses in this thread).

Solution for plain WebDAV/Laverna: One possible solution would be a custom hand-rolled library. The effort to create, fix and maintain such code would certainly dwarf the other problems I listed, and yet it's of course very possible to do it. The main challenge I would assume, apart from literally a thousand other things to implement, is probably variable behavior of WebDAV servers in the end, esp. their Etag implementation. However, if you know of such a library (I know of none, and I also couldn't find one, when I checked again just now), it would become very possible to solve the other outstanding issues in some way and provide WebDAV to technical users.

Please excuse and point out any mistakes or wrong facts contained in this list. I basically wrote all of it off the top of my head. I'll transfer this to the RS wiki sometime soon, and ask some people to review/edit what I wrote, as I think that this is actually a rather useful explanation of the differences between WebDAV and RS, and how RS solves things that WebDAV hasn't, when it comes to client-side Web apps that run entirely in the browser (which is understandable, as the Web platform capabilities didn't really exist back then and the whole concept of offline-first web apps wasn't a thing).

@technodrome If you want to progress your goal of getting WebDAV support, then pointing out potential solutions to any of these problems (or facts that void them entirely) would probably go a long way.

Sorry, but if you cannot understand common sense user perspective because you feel it hurts your feelings or opinions, then the problem is with you. The software intended for people is by definiton used by more than one person - not just you. As such, everyone should be invited to offer an opinion because brainstorming and ideas is what pushes projects forward. And judging by the people issuing +1 for webDav or other standardized protocol support, it's not just me.

You're trying to twist what I said and take things personally even though they were meant as a personal opinion on a general level. But to the point - if a developer doesn't offer any standardized method of saving the user data, then yes - he/she is strong-arming the user to use the dev's preference.

Tech-savvy users does not equal programmers. Do not leap to conclusions just because it fits your rhetoric.

Also, I clearly said I appreciated the work and effort and I truly do. It just does not mean I will blindly agree with everything just because you said so. It is my right as it is yours and there is million ways how to go about solving this issue. Of course you're pushing your product but let's be a bit democratic here and offer a freedom of choice which suits more people, not just you.

While you listed a boatload of reasons why it cannot work and why your RS solution is the cure for every ache, I am sure an intermediary endpoint (Go app?) running as a local daemon (client, server) could take care of authorization and enable a host of possibilities to integrate just about any protocol you want. It surely brings less headache than maintaining a whole stack of technology, which may or may not be forgotten and obsolete in two years.

Also, my comment was not intended to prove anyone wrong, just to say there is a certain industry standard and there is a reason why things become such a standard. If you cannot understand that, I am afraid there is no argumentation that could convince you of any other truth than yours.

Well, I read good arguments on both sides. Eventually owning my storage was a requirement for me, and went to turtl as (encrypted) storage on the server is part of the core of the project.
As mentioned earlier, if a product does not suit all your needs, and in that case there seem to be reasons for it (that of course one could still argue with), well then look for something closer to your needs and requirements.
I'm sure many people are fine with hosting their file on Dropbox. No need to get all worked up, especially on an open source product. I understand and share the frustration though, but open source has many business plans, and sometimes they don't meet our requirements. Just have to deal with it.

Le 24 février 2017 12:27:33 GMT+00:00, technodrome notifications@github.com a écrit :

Sorry, but if you cannot understand common sense user perspective
because you feel it hurts your feelings or opinions, then the problem
is with you. The software intended for people is by definiton used by
more than one person - not just you. As such, everyone should be
invited to offer an opinion because brainstorming and ideas is what
pushes projects forward. And judging by the people issuing +1 for
webDav or other standardized protocol support, it's not just me.

You're trying to twist what I said and take things personally even
though they were meant as a personal opinion on a general level. But to
the point - if a developer doesn't offer any standardized method of
saving the user data, then yes - he/she is strong-arming the user to
use the dev's preference.

Tech-savvy users does not equal programmers. Do not leap to conclusions
just because it fits your rhetoric.

Also, I clearly said I appreciated the work and effort and I truly do.
It just does not mean I will blindly agree with everything just because
you said so. It is my right as it is yours and there is million ways
how to go about solving this issue. Of course you're pushing your
product but let's be a bit democratic here and offer a freedom of
choice which suits more people, not just you.

While you listed a boatload of reasons why it cannot work and why your
RS solution is the cure for every ache, I am sure an intermediary
endpoint (Go app?) running as a local daemon (client, server) could
take care of authorization and enable a host of possibilities to
integrate just about any protocol you want. It surely brings less
headache than maintaining a whole stack of technology, which may or may
not be forgotten and obsolete in two years.

Also, my comment was not intended to prove anyone wrong, just to say
there is a certain industry standard and there is a reason why things
become such a standard. If you cannot understand that, I am afraid
there is no argumentation that could convince you of any other truth
than yours.

--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/Laverna/laverna/issues/6#issuecomment-282279742

--
Envoyé de mon appareil Android avec K-9 Mail. Veuillez excuser ma brièveté.

As such, everyone should be invited to offer an opinion because brainstorming and ideas is what pushes projects forward. And judging by the people issuing +1 for webDav or other standardized protocol support, it's not just me.

Yes! I stated exactly that and then spent time and effort to point out what I think would need to be solved in order to fulfill your wish. I didn't twist anything to "fit my agenda", and I actively invite you and anyone else to point out any errors contained in my objective analysis and I also invited you to help progress the implementation of your own preference/idea.

Unfortunately, it's not easy to find substantial arguments or new facts among the accusations in your follow-up comment. But one paragraph contains at least an idea:

I am sure an intermediary endpoint (Go app?) running as a local daemon (client, server) could take care of authorization and enable a host of possibilities to integrate just about any protocol you want

So yes, that would of course be possible in theory. The problem with that is that it utterly conflicts with your own opinions and argument in the very same paragraph:

It surely brings less headache than maintaining a whole stack of technology, which may or may not be forgotten and obsolete in two years.

Your idea would mean exactly that: create, test and maintain a whole new stack of technology ("client, server") in order for the web app to be able to store data remotely, and sync it across devices. Apart from the problem of inventing and maintaining this whole new stack, it would also be rather difficult to run that local program on all your devices, most importantly mobile ones.

Also, my comment was not intended to prove anyone wrong, just to say there is a certain industry standard and there is a reason why things become such a standard.

I completely agree with that! And I explained at length why WebDAV has indeed not become that standard for client-side web apps, and why the RS protocol was created --initially with WebDAV as API even-- as a new standard to make per-user, self-hosted storage possible for this exact type of applications.

It's actually a rather simple protocol and the opposite of a "technology stack". In case you haven't seen the quick explanation of its parts, you can find it on this website. -- Also, regarding "webDav or other standardized protocol support", that's exactly what RS is meant to be.

Now, you may of course feel negatively about WebDAV not being an overly viable solution for client-side web apps, but that doesn't somehow change the facts at hand, or make me an egotistic asshole for pointing them out. Please, please, let's discuss this rationally, and keep personal accusations out of it. I'm really only trying to help, and there's absolutely no need for negativity. Again: I'm actually trying to help you achieve your goal, and I'm definitely not arguing for RS to be an exclusive solution at all. If anyone can come up with any another solution, that achieves the same goals, then I'm the last person to argue against it based on personal preference or ideology.

Laverna, as a web application, can only support HTTP as a transfer protocol and requires CORS for anything to work. Yet for some reason people are proposing things like BitTorrent Sync or WebDAV, one of which doesn't run over HTTP and neither of which has CORS as part of its specification. In fact the former doesn't even have a specification, it's a proprietary protocol. @technodrome even goes as far as calling the usage of WebDAV "common sense user perspective", though, again, using WebDAV from an in-browser Javascript application just doesn't work in the general case and therefore makes no sense, regardless of perspective. The talk about using an intermediate proxy to add CORS headers also doesn't match my idea of good UX.

I think this really shows that this discussion is mostly dominated by people wishing for "general software freedom" and throwing out the names of some established protocols, not people who have actually seriously looked into any of those options from a technical perspective and determined if they are the right tool for the job. Again I'd like to call out @technodrome, who in addition throws a tantrum because he (understandably) perceives @skddc's advocacy for remoteStorage as shilling pushing his own agenda, at the same time @skddc is the first one in this thread to make a decent argument for the protocol he is proposing.

Perhaps we should just admit that even after the three years this issue has been opened, there is just no standard for this. Which leaves us with three options:

1. Bet on remoteStorage, not-yet-a-standard
2. Use WebDAV anyway, but require CORS headers. Effectively this means only support for NextCloud and ownCloud.
3. Write a custom NextCloud plugin, with custom API

Ad 1: remoteStorage as a protocol is quite good, but I don't think the client library is there yet. In fact the last experience I had with remoteStorage.js (the client library) wasn't good. Everything sort of worked, yet there were still minor bugs everywhere and I just don't consider the high-level "data modelling" API to be usable. The last part shouldn't matter though, given that you can just write your own data model and read and write raw files.

Ad 2: This totally works, but now you have limited yourself to NextCloud/ownCloud while having to deal with the monstrous complexity of WebDAV. I think I can speak for everybody who ever worked with that protocol when I say that usage of WebDAV just for the sake of using a standard is not worth it if the user doesn't benefit from it.

Given that Laverna already has remoteStorage support, I do suppose that investing more time in fixing the rs.js client library is a good way to go forward, though we probably need NextCloud support for anybody to use that backend. A specialized NextCloud app is also a good option IMO, though that misses the goal of having a standard by even more.

Hello there,
Please report to https://github.com/Laverna/laverna/issues/971#issuecomment-411423965 which explain the state of this project.

Have a nice day/night,
Cheers,
Nissar