[Xerte-dev] Re: Lost Data

Thu Feb 6 15:00:29 GMT 2014

Yes, but only if I'm right... would be great to be able to reproduce...

I have a rough list for the new editor and copy+paste and history were too things that I was hoping we could implement...

I've got live multi editing down too (I'm thinking Google Docs here) but might be a step to far ;-)

Regards,

John Smith
Learning Technologist
School of Health & Life Sciences
Glasgow Caledonian University

From: xerte-dev-bounces at lists.nottingham.ac.uk [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Julian Tenney
Sent: Thursday, February 06, 2014 2:51 PM
To: For Xerte technical developers
Subject: [Xerte-dev] Re: Lost Data

If that's not hard to do, that would at least allow us to trap any occasions when this happens...

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Smith, John
Sent: 06 February 2014 14:29
To: For Xerte technical developers
Subject: [Xerte-dev] Re: Lost Data

If thinking a bit of code in the editor that validates what it has loaded?

Like the routine we added to solve the false Upload OK messages - we upload and then check for the existence of a file...

What if, after loading into the editor, the editor sends it back out to a php script which checks it against what is on the server - if they don't match then we have a parsing issue... that is assuming that I am right - I have been known to be wrong on occasion...

Regards,

John Smith
Learning Technologist
School of Health & Life Sciences
Glasgow Caledonian University

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Julian Tenney
Sent: Thursday, February 06, 2014 2:05 PM
To: For Xerte technical developers
Subject: [Xerte-dev] Re: Lost Data

> The fact that a complete node is missing and that the xml is valid makes me think that the editor has not loaded it properly and then saved back out what it had parsed correctly, minus the missing node...

You know a lot more about the sessions thing than I do, so that is reassuring. If what you say is right (I think it might be) then the answer would be to not save and not publish, though of course a save is created when you click play - so how to preserve the good xml which would be on the server?

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Smith, John
Sent: 06 February 2014 13:59
To: For Xerte technical developers
Subject: [Xerte-dev] Re: Lost Data

I don't think the session bug can cause this... if the session drops then the save.php will not save the data that has been posted to it...

It also checks the length of what it has received with what it expects and shouldn't save if different... was this not introduced to solve the truncating xml?

The fact that a complete node is missing and that the xml is valid makes me think that the editor has not loaded it properly and then saved back out what it had parsed correctly, minus the missing node...

Regards,

John Smith
Learning Technologist
School of Health & Life Sciences
Glasgow Caledonian University

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Julian Tenney
Sent: Thursday, February 06, 2014 1:16 PM
To: For Xerte technical developers
Subject: [Xerte-dev] Re: Lost Data

> I still wonder if it's tab nav and/or tab nav+ related too - were the examples you've seen using those pages?

I don't think so, at least, I hadn't put any of those pages in her prototype.

I have a feeling it is related to the Firefox session thing, and somehow getting old data? Is that possible? There is a bug where if you empty the recycle bin, sometimes your workspace refreshes with someone else's LOs in it. My user was using Chrome, but due to the recent problems there had switched to Firefox. Our other user has been reluctant to share any further information unfortunately, so I'm not sure which browser was being used there.

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Ron Mitchell
Sent: 06 February 2014 11:16
To: 'For Xerte technical developers'
Subject: [Xerte-dev] Re: Lost Data

Or change the publish message? e.g. you only need to click yes to commit your changes to the versions others will see. Again I wonder if we change the publish reminder for all we'll get a lot of enquiries saying I've made changes but my learners aren't seeing them. Again seems like changing for all when this is a rare and minority issue.

Agreed it's probably not the best idea to alert users to a problem that is so intermittant.

I think we have had a broken xml problem in previous versions (occasionally) and it certainly seems like connectivity problems can cause this judging by the problems Alistair had but this lost pages without broken xml issue ssems slightly or completely different.

I still wonder if it's tab nav and/or tab nav+ related too - were the examples you've seen using those pages?

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Julian Tenney
Sent: 06 February 2014 10:49
To: For Xerte technical developers
Subject: [Xerte-dev] Re: Lost Data

I'm also wary of suggesting we might have a problem that we might not actually have! Most users aren't doing any funky html stuff, so I don't think most users are at risk, if there is something not escaping properly. Also, most users aren't on 2.1, which is where I think this problem might be (we've never seen it before 2.1) and even if we add some safety code, there will be a lot of older installs out there without it.

One thing that I wonder is whether to remove the 'do you want to publish' dialog, because people just get in the habit of saying 'yes' and then over-write the data.xml with the duff preview.xml, and then you've got no way back: every single time lost data has cropped up, my first question is 'did you publish the file' and the answer every time is 'yes'. Doh.

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Ron Mitchell
Sent: 06 February 2014 10:15
To: 'For Xerte technical developers'
Subject: [Xerte-dev] Re: Lost Data

Yeah I know what you mean but I was thinking of then prompting users what to do when it happens.

But yes a history of edits of some kind sounds good but I can't imagine how that would work e.g. any single edit could cause the loss

So thinking out loud perhaps each time a change is made a copy of preview is also created but up to a max of say 5 then rotated. All users are briefed that if they ever notice a problem when editing/previewing to stop editing at that point and notify their sys admin with details.

I wonder if before doing any work on this we need to send a message to the various mailing lists etc asking for specific details if and when it happened? Or would that cause a mis-placed concern given this is a rare occurrence?

I just wish we could reproduce or had a definitive example :-(

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Julian Tenney
Sent: 06 February 2014 09:48
To: For Xerte technical developers
Subject: [Xerte-dev] Re: Lost Data

I think you'd end up backing up the broken file. It's not really a backup we need, it's a history of the edits?

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Ron Mitchell
Sent: 05 February 2014 16:57
To: 'For Xerte technical developers'
Subject: [Xerte-dev] Re: Lost Data

The problem also is that it's not 100% clear if this happens only after the change that caused the break. So even if we got hold of a restored backup it probably wouldn't help.

At some point but probably not until middle of next week I'm going to see if I can automate my own local back up of the user-files folders on one or two of the Techdis installations. At the moment the backups are off-server and I don't have access. If I can automate local backups at least I can quickly get to the working xml as soon as someone reports this.

Cheers
Ron

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Julian Tenney
Sent: 05 February 2014 16:16
To: For Xerte technical developers
Subject: [Xerte-dev] Re: Lost Data

No idea. In the case here the html was all escaped in CDATA, being part of the main page text - I wrote it to do some custom stuff, which involved adding a custom button to each page to display a transcript for narration and a couple of other things. Once I'd sorted it out, I made a prototype and gave it to the author to duplicate, with a set of pages set up to duplicate so she could crack on with the content quickly. I didn't experience any problems myself.

I suspect in both cases, the user has limited html skills. They could easily have left a tag unclosed, or an attribute without a quote, the sort of thing that I'm (a bit) less likely to do.

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Smith, John
Sent: 05 February 2014 16:10
To: For Xerte technical developers
Subject: [Xerte-dev] Re: Lost Data

Hi,

I suspect that the custom HTML is your best link to reproducing this... something in there has probably caused the nodes not to load properly into the Flash editor and then when it was written back out those nodes were then overwritten. This is the problem with mixing XML and HTML, unless its properly wrapped in CDATA blocks.

Do you have any clue as to what was in the HTML?

Regards,

John Smith
Learning Technologist
School of Health & Life Sciences
Glasgow Caledonian University

From: xerte-dev-bounces at lists.nottingham.ac.uk<mailto:xerte-dev-bounces at lists.nottingham.ac.uk> [mailto:xerte-dev-bounces at lists.nottingham.ac.uk] On Behalf Of Julian Tenney
Sent: Wednesday, February 05, 2014 3:59 PM
To: For Xerte technical developers (xerte-dev at lists.nottingham.ac.uk<mailto:xerte-dev at lists.nottingham.ac.uk>)
Subject: [Xerte-dev] Lost Data

Hi,

Just wondering if anyone else has experienced the same data loss we had here last week, and that was reported on the Xerte list yesterday. In both cases, the user reported 'lost pages'. Looking at the xml it was intact, with a closing </learningObject> tag: that's the first time I've ever seen data loss with a valid xml file left over (I would expect it to truncate at the point the file broke, and safety code should stop that then saving, right?)

Maybe it's just coincidence, but I'm a bit worried there might be a problem we don't know about causing data to get lost. In the case reported here, I don't suspect (highly) user error (it's always a possibility). In the case on the Xerte list I'm less certain. I've asked for more info, but none has been forthcoming.

Obviously we need to try and reproduce it if possible, or put in place some sort of safety mechanism. I did notice that both had some custom HTML going on in the code. Any ideas or insight?

Julian

This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it.   Please do not use, copy or disclose the information contained in this message or in any attachment.  Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham.