Signature_pad: Capturing biometric data with the sign

It depends on what data you exactly have in mind, but I think it would be possible. There's ISO/IEC 19794-7:2014 that "_specifies data interchange formats for signature/sign behavioural data captured in the form of a multi-dimensional time series_", so it would be awesome to use this standard instead of coming up with our own. Unfortunately, I don't have access to this document and don't know how the format actually looks like :/

Thanks a lot for your answer! I will look forward for that standard and
I'll report here any advance I could get for further use.

El 21/1/2017 10:21, "Szymon Nowak" notifications@github.com escribió:

It depends on what data you exactly have in mind, but I think it would be
possible. There's ISO/IEC 19794-7:2014
http://www.iso.org/iso/catalogue_detail.htm?csnumber=55938 that "specifies
data interchange formats for signature/sign behavioural data captured in
the form of a multi-dimensional time series", so it would be awesome to
use this standard instead of coming up with our own. Unfortunately, I don't
have access to this document and don't know how the format actually looks
like :/

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/szimek/signature_pad/issues/213#issuecomment-274250329,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APdPAIzn5SZW2nwNKDD3evH4ijb_1Mk7ks5rUc4wgaJpZM4LphIH
.

@kalihos You can also try out the latest beta release (1.6.0-beta.1), which has #toData method that returns all points grouped in lines, but currently it returns only time, x and y coordinates for each point. It would be rather easy to add velocity and acceleration to it, but I don't think we can capture anything else.

You can try out this beta version on the demo page - just draw anything and then call signaturePad.toData() in dev tools console.

Of course, capturing velocity and acceleration would be great, and enough
to legally warranty that the signer can't reject the authority. Anyway, the
new #toData method is an excellent notice for me. I will try and feedback
you.

Thanks a lot!

2017-01-22 14:35 GMT+01:00 Szymon Nowak notifications@github.com:

@kalihos https://github.com/kalihos You can also try out the latest
beta release (1.6.0-beta.1), which has #toData method that returns all
points grouped in lines, but currently it returns only time, x and y
coordinates for each point. It would be rather easy to add velocity and
acceleration to it, but I don't think we can capture anything else.

You can try out this beta version on the demo page
https://szimek.github.io/signature_pad/ - just draw anything and then
call signaturePad.toData() in dev tools console.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/szimek/signature_pad/issues/213#issuecomment-274331306,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APdPAH0oBwyertm7moGPS24Fubi33MKOks5rU1sJgaJpZM4LphIH
.

I suppose acceleration and velocity could be inferred from the timestamps - but, for what it's worth, it would be really nice having an option to include those data points when calling toData, or to have a new method to impute them.

By the way - I'm loving the beta. Have been testing it out a bit and I look forward to being able to store the raw data in the DB in the future. Great stuff! 👍

@JD-Robbs

By the way - I'm loving the beta.

Thanks! :)

Have been testing it out a bit and I look forward to being able to store the raw data in the DB in the future.

That's exactly why I'd prefer to use some existing standard for storing this stuff. Once people start storing it in their DB it will be hard to modify it in the next versions. Originally, it was meant to be private and used only to draw SVG and help with unit tests.

I'm just wondering what velocity and acceleration should be for the first point in each line - 0 or null/undefined?

I've tried your excellent beta version, and it works great! As you said, it would be fantastic to have in addition velocity and acceleration available. In the legal point, I think it's not relevant having 0 or null/undefined for the first point, because it would be only used to compare a sign with another previously saved one, so anyone can check if it's the same person who signed both times. But, technically speaking, it would maybe be better to set acceleration for first point to 0. I think it would be easier to store the full array, because a 0 is always an easier format to export to a file... otherwise we would store "null-5-3-7-2...", and it sounds a bit confusing in my opinion...

Respect to the standard, as I said, I'm trying to get detailed information about it, but the high legal differences in complexity between different countries doesn't help at all... For example, in Spain there is no legal certification about this kind of "digital-handwritten" sign, but it's accepted. But, in contrast with traditional handwritten sign or digital-certificated sign, here you are the one who need to demonstrate that your process is ok and you can ensure the authenticity of the sign, but based on a Spanish legal order (http://noticias.juridicas.com/base_datos/Admin/l59-2003.html - sorry, only Spanish).

Hi again!

I can't access anyway to the standard document, but I've been looking at the Spanish Law, and your currently captured data is more than ok. Only data such as pressure points and inclination are impossible to catch from a normal device, but they are not fully necessary. Anyway, velocity and acceleration would be amazing to ensure user's not rejecting his own sign. I'm starting my development integrating your component. If you add such additional data in the near future, I will try to adapt the data in my database. It's working really great...

Thanks a lot!

I guess if a point already has velocity and acceleration, then it doesn't really need time anymore. I'll try to update it, though it might take a while. Maybe someone wants to create a PR? 😉

@szimek

That's exactly why I'd prefer to use some existing standard for storing this stuff. Once people start storing it in their DB it will be hard to modify it in the next versions. Originally, it was meant to be private and used only to draw SVG and help with unit tests.

That's a very good point - I haven't even thought that far with all my excitement (although I'll still be storing BLOBs, it's nice to have the raw biometric information as well). I suppose any future version will still be based on the basic information around coordinates and time - therefore, I guess, there could always be an upgrade/conversion path should things change (e.g. measures/scales). But, then again, I'm not really up-to-speed on how everything is implemented.

I'm just wondering what velocity and acceleration should be for the first point in each line - 0 or null/undefined?

I guess velocity would be 0, and acceleration the delta to the second point? Although, then, what would be the acceleration of the last point? Perhaps, both should be 0 - not sure.

I guess if a point already has velocity and acceleration, then it doesn't really need time anymore. I'll try to update it, though it might take a while. Maybe someone wants to create a PR? 😉

Alternatively, I think the toData and fromData behaviour could just remain as-is (just to keep the required data to the bare minimum).

Instead, there could, perhaps, be an additional method on the prototype which can turn the output of toData to {x: number, y: number, time: number, velocity: number, acceleration: number}[].

Maybe you are right. I could infer that information starting from time,
thought it could be easier to compare two signs. But yes, it would overload
the object without needing. I will work with your current beta version.
It's perfect as-is!

El 24/1/2017 18:20, "Szymon Nowak" notifications@github.com escribió:

I guess if a point already has velocity and acceleration, then it doesn't
really need time anymore. I'll try to update it, though it might take a
while. Maybe someone wants to create a PR? 😉

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/szimek/signature_pad/issues/213#issuecomment-274872842,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APdPAG5bKRA4vv9_uqT563KiR1eqpZ_hks5rVjLCgaJpZM4LphIH
.

+1 on this.
This is a good info to accomplish the ISO 19794-7.

@javenosa Do you have any info what data/format ISO 19794-7 actually specifies?

X coord from pen (increments when moves to right)
Y coord from pen (increments when moves to top)
Z coord from pen (increment when moves the lift between the device and the pen, this doesn't apply here)
Horizontal velocity (difference between velocities)
Vertical velocity (difference between velocities)
Horizonal acceleration
Vertical acceleration
Time
Time difference
Pressure
Angle (this doesn't apply here)

With your plugin you can get the most items from this list.

https://image.slidesharecdn.com/onlinesignaturerecognition-130201011136-phpapp02/95/online-signature-recognition-16-638.jpg?cb=1359681345

@javenosa Thanks! If there's anyone who wants to do PR to add horizontal and vertical velocities and acceleration and time difference, it would be awesome 😉 Like you said we can't really get Z coord, pressure or angle.

One difference is that in HTML canvas Y coord increases when moving to the bottom. I'm not sure if it makes sense to change it to comply with the spec, because it will be a bit annoying to use this data then to draw on a canvas and you'd probably need to know height of the canvas to know what to subtract from...

I'm also wondering if we should also have some other method (or an option for #toData) that returns pen color and max/min widths per line as well. It's not really necessary for signature biometric data, so maybe it can be added eventually later, if enough people complain about it.

I think pressure can be get on the correct device (of course, with pressure support, iPhone 6s, iPhone 6s Plus, iPhone 7, iPhone 7 Plus, etc).

I had no idea that it's possible to get these values in JS, that's pretty awesome.

Though looking at https://pressurejs.com source code it doesn't seem to be trivial to implement and I'm not sure it's worth the additional code, so maybe it can be postponed for now and eventually added in the next versions.

Yep, pressurejs is one alternative (I just copy the iphone's version from their site hahah).
I understand your point, sound logic.

I tested pressurejs, and it seems to say always that device is not supported (I tried with normal Macbook trackpad, iPhone...). Anyway, looking at the spec information from @javenosa, I think we're are currently storing every necessary information into DB with toData method, except maybe pressure (difficult to get without a Wacom device or similar), and angle (similar to previous). I'm currently using your excellent plugin into my application, and it works fine. And, if I need later to analyze any signature, I can read biometric information and get additional data like velocity or acceleration. I think it's not necessary to store redundant information...

@kalihos If we decide to skip the pressure for now, then you're right that all other data (i.e. velocities and accelerations) can be derived from time. Maybe for now it would be easier to just return time only and provide a script in the README file that can convert it to something that has this additional data.

I'm just wondering if we also need to return size of the canvas (or maybe better max/min X and Y coords), so that it's possible to reverse Y coords (0 at the bottom vs 0 at the top).

@szimek It's a good idea to provide max/min X and Y coords. But take in account that, when saving to SVG, such information could be confusing due to resizing possibility. Don't you think it would be better to store the signature only in PNG format if you want to store max X/Y?

Respect to the script in the README file, yes, it's another good idea. It would be only necessary to ensure a correct comparison for legal purposes (legal claim).

@kalihos I was thinking about storing max/min X and Y only so that it's possible to reverse Y for biometric data. As @javenosa mentioned before, in the spec 0 is at the bottom, but in a canvas 0 is at the top. If we'd like to follow the spec, I'd need some way to "reverse" Y coords. Of course max/min X and Y can also be derived by iterating over all points returned by #toData method, so it doesn't necessarily need to be returned by #toData itself.

@szimek It's a good option to store max/min X and Y, so you can restore the full canvas, and you can accomplish the spec. Anyway, I don't think it's fully necessary to take the spec literally, if you can ensure the signature authenticity... but it's right to store it to be more "pure" ;)

Any update on this?
Acceleration and velocity would be an amazing feature to the "toData" method.

Thanks

@javenosa I'm currently focusing on releasing 1.6 with SVG support, but don't have much free time, so it's progressing rather slowly.

if you want to see it in 1.6, PRs are welcome ;) Also, it should be quite easy to write a simple script that takes output from toData and adds velocities and acceleration - all you need to calculate those are time diffs, which are already returned.

I don't know how be the formulas to calculate the velocity and acceleration, if you can tell me that, I can do a PR.

velocity = change in distance / change in time
acceleration = change in velocity / change in time

There's Point#velocityFrom(point) that will give you velocity between 2 points and once you got that you can use almost the same code to calculate acceleration:

// This one already exists
Point.prototype.velocityFrom = function (start) {
  return (this.time !== start.time) ? this.distanceTo(start) / (this.time - start.time) : 1;
};

// This one should be added
Point.prototype.accelerationFrom = function (start) {
  return (this.time !== start.time) ? this.velocityFrom(start) / (this.time - start.time) : 1;
};

Both methods should probably return 0 instead of 1 in case the times are the same, because the position should be the same as well...

Interesting project! The ISO biometry standard is an ASN-1 encoded binary standard which we were about to implement. Luckily for us an XML version was released in 2015 which is much easier to implement.

Regarding the post from javenosa it is worth mentioning that most of the channels (they are called channels in the ISO standard) are optional. Only x,y and t are required and the rest is optional. So it may be nice to have pressure or Z-coords, angle etc. but it is not mandatory for a fully compliant biometric dataset. Additionally many other values can be calculated later from the existing x,y, and t values like (velocity, acceleration etc.). So I would start with a most basic dataset: x,y,t ...

@Lonzak If anyone can provide info how exactly the data structure should look like compared to what it looks like now, it would be great. Because currently #toData method returns x, y coords and time, so I guess it's just a matter of transforming the current structure to whatever is expected by the ISO standard.

Yeah no problem: Have a look at the schema and the supplied example. For XML<->JS conversion you could use jsonix. But as already mentioned this is the newer XML based standard, the older ASN-1 based version is much more complex to create imho (DER, BER encoder with support for encoding instructions. We didn't find any open source encoder/decoder which supported that)...

@szimek awesome library, using it in a react native project and it's smooth as butter!
@kalihos do you know of a good signature recognition/matching sdk?

Sorry, but I haven't got anything. I only use it for low-risk documents, so
I don't expect anyone to reject his own sign.

El 21/9/2017 23:21, "Mark Vayngrib" notifications@github.com escribió:

@szimek https://github.com/szimek awesome library, using it in a react
native project and it's smooth as butter!
@kalihos https://github.com/kalihos do you know of a good signature
recognition/matching sdk?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/szimek/signature_pad/issues/213#issuecomment-331285500,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APdPANXnge-rnI2TRuV9w-uqdp8kn_nbks5sktNEgaJpZM4LphIH
.

@szimek I've found something like that:
http://stg.jsm.gov.my/documents/372014/372056/MS+ISOIEC+19794-7+2008+-+INFORMATION+TECHNOLOGY+BIOMETRIC+DATA+INTERCHANGE+FORMATS+PART+7+SIGNATURE+SIGN+TIME+SERIES+DATA-712359.pdf/25a168da-0654-4907-b527-ad808005fe78?version=1.0&previewFileIndex=

@mgierw Thanks! However, now that there's SignaturePad#toData, one could (probably) write a tiny script that takes output of this function and converts it to the format expected by this standard. It wouldn't even have to be a part of this library.

Something like:

const data = convertToISO(signaturePad.toData());

If someone comes up with such library, I'd add info about it to the README file.

Hi,
this conversation is really interesting!
did someone tried to implement the ISO standard?
I would like to help

Any news on this?

As you were talking about speed and accelleration, what about force? There is a library called pressurejs, which allowes you to meassure the force of the touch on devices that support it. I am not in this topic, so I don't know anything about the legal part.

@Springrbua v3 of this library supports pointer events, which can provide such info if a device provides it. So it should be possible, but I don't plan it for the first 3.0.0 release, because of lack of time and devices to test it on.

Thanks for the fast answer. I just discovered pressurejs lately so I was curious, if it could be used for digital signatures too.

@szimek awesome library, using it in a react native project and it's smooth as butter!

@mvayngrib How exactly did you manage to make it work with RN?

@ntelkedzhiev I'm using react-native-signature-pad which is a wrapper around this library. Or rather, I'm using a fork of that wrapper, as I wanted to get width/height back along with the data url (https://github.com/tradle/react-native-signature-pad)

@mvayngrib Thanks for letting me know! I'm doing the same thing. I also tried updating https://github.com/kevinstumpf/react-native-signature-pad to use the new updated version of this repo but it will take more time than I have right now. I think all modules have to be put inside one file and they should be in CJS/UMD format.

any updates on this topic?

Hi, I found a biometric implementation standard.
https://github.com/biosignin/bsi-core
It is a bit old but it can be worth as a starting point

Hi! This would be such a great addition to the project. Any news about it?

you mean like pulse, blood pressure, pupil dialation, body temperature?

On Tue, Sep 15, 2020 at 1:24 AM corrrso notifications@github.com wrote:

Hi! This would be such a great addition to the project. Any news about it?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/szimek/signature_pad/issues/213#issuecomment-692553010,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AK7MVVQPIVBQFFOLU3NGV4TSF4QCBANCNFSM4C5GCIDQ
.

--
here's my signature

you mean like pulse, blood pressure, pupil dialation, body temperature?
…
On Tue, Sep 15, 2020 at 1:24 AM corrrso @.*> wrote: Hi! This would be such a great addition to the project. Any news about it? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#213 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7MVVQPIVBQFFOLU3NGV4TSF4QCBANCNFSM4C5GCIDQ .
-- here's my signature

He means stroke pressure, acceleration, speed, etc. as defnied in ISO 19794-7 specifications.
This is considered biometric data useful to legally validate a signature, the signature image alone is insufficient in some countries like Italy and maybe Spain.

you mean like pulse, blood pressure, pupil dialation, body temperature?
…
On Tue, Sep 15, 2020 at 1:24 AM corrrso _@_.*> wrote: Hi! This would be such a great addition to the project. Any news about it? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#213 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7MVVQPIVBQFFOLU3NGV4TSF4QCBANCNFSM4C5GCIDQ .
-- here's my signature

He means stroke pressure, acceleration, speed, etc. as defnied in ISO 19794-7 specifications.
This is considered biometric data useful to legally validate a signature, the signature image alone is insufficient in some countries like Italy and maybe Spain.

Yea. It's is true in spain is a requirement for advanced digital signature.

I try to do in https://github.com/javyxx/signature_pad is simple modification to save pressure

https://developer.mozilla.org/en-US/docs/Web/API/PointerEvent/pressure

Hi...

If someone is interested, I made in my fork a basic implementation to capture pressure when available and also I implemented a function to convert the captured data to the ISO 19794-7 specification.

Any comments, tests, fixes or improvements are welcome.

May be very interesting to merge with this project anyway

I forked your repo to update with latest changes in szimek original repo.
I think that is interesting to add the accelerations AX and AY, that can calculate like the velocity. (Increment of the velocity / increment time)
What is your opinion?

What is your opinion?

Hi Certifirm...

I tried to add the accelerations also, but I was having problems with the validation of the xml data results. The ISO specify a maximum value and sometimes I was having values bigger than that maximum. There is a way in the ISO that allows you to scale up and down the values, but it wasn't too much clear for me and I didn't have more time to investigate. Sorry.