Storing iPhone apps locally with data URLs

« Good Hash Functions - Unspecified Attributes »

Storing iPhone apps locally with data URLs

By W. Clawpaws on Monday 16 July 2007, 10:11 - iPhone - Permalink

Some people think that you need net access to run web-based applications on your iPhone. Not so. The URL below provides a simple tip calculator (sadly this crappy blogging system doesn't let me do a direct link, which sucks, but you can copy and paste and add it your bookmarks on your computer then sync with your iPhone, and/or make your own page with a direct link). By using a data: URL, the entire page content is all in the URL. If save a bookmark for this URL, you can access this little JavaScript-based app even in airplane mode.

data:text/html;charset=utf-8;base64,PGh0bWw+CjxoZWFkPgo8bWV0YSBuYW1lPSJ2aWV3cG9ydCIgY29udGVudD0id2lkdGggPSAyNDAiIC8+Cjx0aXRsZT5UaXAgQ2FsY3VsYXRvcjwvdGl0bGU+Cgo8c2NyaXB0PgoKZnVuY3Rpb24gdGlwKGFtb3VudCkgewogICAgcmV0dXJuIHRpcDsKfQoKdmFyIG91dHB1dCA9IG51bGw7CnZhciBwZXJjZW50ID0gMTguNSAvIDEwMDsKdmFyIHJ0aXBfZmFjdG9yID0gMC4yNTsKdmFyIHJ0b3RhbF9mYWN0b3IgPSAxLjAwOwoKZnVuY3Rpb24gd3JpdGVPdXQobGluZSkgewogICBpZiAob3V0cHV0KSB7CiAgICAgIG91dHB1dC5hcHBlbmRDaGlsZChkb2N1bWVudC5jcmVhdGVFbGVtZW50KCJiciIpKTsKICAgfSBlbHNlIHsKICAgICAgb3V0cHV0ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoIm91dHB1dEFyZWEiKTsKICAgfQogICBvdXRwdXQuYXBwZW5kQ2hpbGQoZG9jdW1lbnQuY3JlYXRlVGV4dE5vZGUobGluZSkpOwp9CgpmdW5jdGlvbiB1cGRhdGUoKQp7CiAgICB2YXIgYW1vdW50ID0gTnVtYmVyKGV2YWwoaW5Gb3JtLm51bS52YWx1ZSkpOwogICAgaWYgKGFtb3VudCA9PSBOYU4pIHsKICAgICAgICB3cml0ZU91dCgnSHVoPycpOwogICAgfQogICAgdmFyIHRpcCAgPSBhbW91bnQgKiBwZXJjZW50OwogICAgdmFyIHJ0aXAgPSBNYXRoLnJvdW5kKHRpcCAvIHJ0aXBfZmFjdG9yKSAqIHJ0aXBfZmFjdG9yOwogICAgd3JpdGVPdXQoJyQnICsgYW1vdW50LnRvRml4ZWQoMikgKyAnICsgJCcgKyBydGlwLnRvRml4ZWQoMikgKyAnID0gJCcgKyAoYW1vdW50K3J0aXApLnRvRml4ZWQoMikpOwogICAgdmFyIHJ0b3RhbCA9IE1hdGgucm91bmQoKGFtb3VudCArIHRpcCkgLyBydG90YWxfZmFjdG9yICsgMC4yNSkgKiBydG90YWxfZmFjdG9yOwogICAgd3JpdGVPdXQoJyQnICsgYW1vdW50LnRvRml4ZWQoMikgKyAnICsgJCcgKyAocnRvdGFsLWFtb3VudCkudG9GaXhlZCgyKSArICcgPSAkJyArIHJ0b3RhbC50b0ZpeGVkKDIpKTsKICAgIAp9CgpmdW5jdGlvbiB6YXAoZmllbGQpIHsKICAgIGlmICghIGZpZWxkLnphcHBlZCApIHsKICAgICAgICBmaWVsZC56YXBwZWQgPSB0cnVlOwogICAgICAgIGZpZWxkLnZhbHVlICA9ICIiOwogICAgfQp9Cgo8L3NjcmlwdD4KCjwvaGVhZD4KPGJvZHk+Cgo8aDE+VGlwIENhbGN1bGF0b3I8L2gxPgoKPHA+CkVudGVyIGFmdGVyLXRheCB0b3RhbDoKPC9wPgo8Zm9ybSBuYW1lPSJpbkZvcm0iIG9uU3VibWl0PSJ1cGRhdGUoKTsgcmV0dXJuIGZhbHNlOyI+CjxpbnB1dCB0eXBlPSJ0ZXh0IiBuYW1lPSJudW0iIG9uRm9jdXM9InphcCh0aGlzKSIgdmFsdWU9IjI0LjM3IiAvPgo8aW5wdXQgbmFtZT0ic3VibWl0IiB0eXBlPSJidXR0b24iIG9uQ2xpY2s9InVwZGF0ZSgpOyIgdmFsdWU9IlRpcCIgLz4KPC9mb3JtPgoKPHAgaWQ9Im91dHB1dEFyZWEiPgo8L3A+Cgo8L2JvZHk+CjwvaHRtbD4=

By putting images inline using data: URLs, you can create pretty rich pages and store them locally. I created a 363,488 byte URL for my home page (complete with images) and it loaded just fine on my iPhone.

Here's a quick Perl one-liner to turn HTML into a data: URL.

perl -0777 -e 'use MIME::Base64; $text = <>; $text = encode_base64($text); $text =~ s/\s+//g; print "data:text/html;charset=utf-8;base64,$text\n";'

By making these links programmatically, you even have an ugly hack to do persistent storage on the iPhone. Just encapsulate your app and its state in its URL.

Comments

1. On Monday 16 July 2007, 13:22 by Raj

Looks like a cool way to use something like http://tiddlywiki.com/

2. On Monday 16 July 2007, 13:35 by pauldwaite

Fantastic. I'm sure I'm being a newb about this, but I find it mind-blowing.

3. On Monday 16 July 2007, 13:59 by Pedro Estarque

Damn, this is a claver one. Congrats!

4. On Monday 16 July 2007, 15:26 by deadesq

For the relatively uninformed--is there a way to take an existing iPhone "app" page and do this quickly? I think you'd have a line out your door of people waiting for a quick way to embed apps in this manner--I'd love to have some games on my iPhone this way.

Thanks!

5. On Monday 16 July 2007, 16:37 by David Lindquist

Here is a bookmarklet that will grab the current page's source, generate a data: URL, and load it into the browser, which can then be saved as a bookmark:

javascript:x=new XMLHttpRequest();x.onreadystatechange=function(){if(x.readyState==4)location='data:text/html;charset=utf-8;base64,'+btoa(x.responseText)};x.open('GET',location);x.send('');

Note that scripts and stylesheets must be embedded in the page, not linked. Also, as you mentioned above, images should be embedded using data: URLs as well.

6. On Monday 16 July 2007, 17:52 by Chris Barthelemy

This is fantastic! I had no idea this was possible. What is the technical limitation (length) of such a URL?

Thanks a lot.
Chris

7. On Monday 16 July 2007, 18:24 by Bruce Collier

Brilliant... very nifty. Cheers for posting this.

I can see Javascript libraries being coded around this feature in no time.

8. On Monday 16 July 2007, 20:57 by overkill

that may help:

http://software.hixie.ch/utilities/...

http://en.wikipedia.org/wiki/Data:_...

9. On Monday 16 July 2007, 21:42 by iluvcapra

"Note that scripts and stylesheets must be embedded in the page, not linked. Also, as you mentioned above, images should be embedded using data: URLs as well."

What we need now is a sortof cross between wget(1) and ld(1), something that will take an HTML file, read the links in it (like "link symbols" in an object file) and resolve them into data, replacing the links with the data at the other end -- essentially "static linking" the linked data into a single page. For many SPAs, it would be enough to simply resolve the <head> links.

10. On Monday 16 July 2007, 22:06 by iluvcapra

One more thought on this: this also works for saving Word Docs and Excel files in bookmarks too, since the iPhone uses plugins to render these in Web Views. All you have to do is use the appropriate mime type for the document you are encoding:

mimetype = case file.split('.').last.downcase
when 'pdf'
'application/pdf'
when 'html' || 'htm'
'text/html'
when 'doc'
'application/msword'
when 'xls'
'application/vnd.ms-excel'
end #case

Further, you can take a web page in safari and save it as a "web archive." Web archives can then be converted to word documents (thus embedding everything) using textutil(1) from the command line (on OS X systems)

11. On Monday 16 July 2007, 23:56 by W. Clawpaws

As far as the maximum length of data: URL Safari will allow, I haven't tested beyond the 350K URL I made for my home page. That strikes me as fairly huge. Clearly Safari (and Firefox) go beyond the requirements of RFC 2397, which claims that data: URLs can only be short because of various sundry HTML limits, namely:

The "data:" URL scheme is only useful for short values. Note that some applications that use URLs may impose a length limit; for example, URLs embedded within anchors in HTML have a length limit determined by the SGML declaration for HTML [RFC1866]. The LITLEN (1024) limits the number of characters which can appear in a single attribute value literal, the ATTSPLEN (2100) limits the sum of all lengths of all attribute value specifications which appear in a tag, and the TAGLEN (2100) limits the overall length of a tag.

One other thing, as I alluded to in my original post, it is really easy to make a program that edits its own data: URL. You can fetch and change the URL via the location variable, and decode/encode it via atob/btoa.

12. On Monday 16 July 2007, 23:59 by ethan

in PHP, use $data = base64_decode($data); to convert to base64 for images.

I'm wondering if this is something that should be implemented for IUI? Embed the framework, AJAX the content?

13. On Tuesday 17 July 2007, 02:14 by hansp

Here is a quick development package, all in the URL ;-)

data:text/html;base64,PGh0bWw+CjxoZWFkPgo8dGl0bGU+RGF0YSBVUkwgRGV2ZWxvcGVyPC90aXRsZT4KPHNjcmlwdD4KdmFyIGdlbmJ1dHRvbiA9ICI8YnV0dG9uIG9uY2xpY2s9XCJkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgndXJpJykuaW5uZXJIVE1MPW1ha2VEYXRhTGluaygnVVJMJywgZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ3RleHQnKS52YWx1ZSk7XCI+R2VuZXJhdGUgREFUQSBVUkw8L2J1dHRvbj4iOwpmdW5jdGlvbiBtYWtlRGF0YUxpbmsgKGxpbmssIGhyZWYpIHsgCiAgcmV0dXJuICc8YSB0YXJnZXQ9Il9ibGFuayIgaHJlZj0iZGF0YTp0ZXh0L2h0bWw7YmFzZTY0LCcrd2luZG93LmJ0b2EoaHJlZikgKyAnIj4nICsgbGluayArICc8L2E+JzsKfQpmdW5jdGlvbiBmZXRjaERhdGFMaW5rIChsaW5rKSB7CiAgdHJ5IHsKICAgIHZhciBtID0gbGluay5tYXRjaCAoL14oKGRhdGE6fGNoYXJzZXQ9KVteO10rOykqKGJhc2U2NCwpPyguKykkL2kpOwogICAgcmV0dXJuIGF0b2IobVs0XSk7CiAgfSBjYXRjaCAoZSkgewogICAgLy8gYWxlcnQgKGUgKyAiXG4iICsgbGluayArICdcbiInICsgbS5qb2luICgnIlxuIicpICsgJyInKTsKICAgIHJldHVybiAnJzsKICB9Cn0gCjwvc2NyaXB0Pgo8L2hlYWQ+Cjxib2R5IG9ubG9hZCA9ICIKICBkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgndGV4dCcpLnZhbHVlID0gZmV0Y2hEYXRhTGluayh3aW5kb3cubG9jYXRpb24uaHJlZik7CiAgZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ3VyaScpLmlubmVySFRNTD1nZW5idXR0b247CiAgZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ3VyaScpLmZpcnN0Q2hpbGQub25jbGljaygpOyI+CjxpbnB1dCBpZD0ndXJsJz4KPGJ1dHRvbiBvbmNsaWNrPSJkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgndGV4dCcpLnZhbHVlID0gZmV0Y2hEYXRhTGluayhkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgndXJsJykudmFsdWUpOwogIGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCd1cmknKS5pbm5lckhUTUw9Z2VuYnV0dG9uOwoiPkRlY29kZSBEYXRhIFVSTDwvYnV0dG9uPgo8aHIvPgo8dGV4dGFyZWEgaWQ9InRleHQiIAogIGNvbHM9IjUwIiAKICByb3dzPSIyMCIKICBvbmtleWRvd249ImRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCd1cmknKS5pbm5lckhUTUw9Z2VuYnV0dG9uIj4KPC90ZXh0YXJlYT48YnIvPjxici8+CjxwIGlkPSJ1cmkiPjwvcD4KPC9ib2R5Pgo8L2h0bWw+

When opening this link, the text area will show the html/script source of the page.

Entering a data url into the upper input element and clicking on the decode button will decode the URL into the lower text area where you can edit it.

When the text area is changed, the URL link below it changes to a Generate button which, when clicked, will convert the text area into a Data URL

Clicking the URL link will open the data URL in a new window.

Developed and tested on Ubuntu/firefox, let me know if there are problems on other systems

Enjoy...

14. On Tuesday 17 July 2007, 13:18 by Attila Csordas

Could this data URL hack be a way to embed html versions of PDF files into Safari and look and zoom them via the browser instead of using the poor email attachment capacity without landscape mode?

15. On Tuesday 17 July 2007, 14:58 by W. Clawpaws

Regarding storing a PDF file in a data: URL, yes, you can. You can encode any kind of content that Safari can render. Whether its worth going to the trouble is another question though.

16. On Tuesday 17 July 2007, 16:40 by iluvcapra

I successfully encoded the entire first volume of Gibbon's Decline and Fall to PDF and moved it to my iPhone. It even opens it, though Safari generally crashes after a few minutes of scrolling down pages. The PDF I made was 1.2 Megs.

Also, Safari on my Macbook would beachball for minutes at a time when I'd try to copy/paste or move the data URL around the bookmark page. I had to use the Property List editor to add the data: URL into my bookmarks with any speed.

17. On Tuesday 17 July 2007, 21:26 by levyesq

OK, guys, for non-programmers, where do you put the code to make it work - in plain English, please. Thanks.

18. On Tuesday 17 July 2007, 23:13 by Attila Csordas

Thanks fellows for the tips, here is the non geek version optimized for PDF storing and reading via Safari. Long live the landscape mode: http://pimm.wordpress.com/2007/07/1...

19. On Wednesday 18 July 2007, 10:05 by GQB

Not to put too fine a point on it, but that's actually a 6-line perl script.

20. On Thursday 19 July 2007, 01:43 by W. Clawpaws

To respond to “GQB”'s claim that my Perl one-liner is a six liner: splitting hairs over this kind of stuff seems pretty silly to me, but hey. FWIW, while it's true my original wasn't one statement, it nevertheless was one (long) line. When I type code into the shell on one line, I (and others) call it a one liner.

In any case, the Perl script would be better written to do all its real work in one statement, as follows:

perl -0777 -e 'use MIME::Base64; print "data:text/html;charset=utf-8;base64,", encode_base64(scalar(<>), ""), "\n";'

So, the original is hardly a model of perfection, but I when wrote it, I was just writing a snippet of Perl in a few seconds at the shell prompt. Good enough was good enough.

21. On Thursday 19 July 2007, 12:15 by Danny Goodman

Bear in mind that apps running via the data: protocol do not appear to preserve cookies. I use a cookie for preferences in my tip calculator, which means in the data: version, you have to reset your preferences (if they're not the default) each time you load the app. I made a data: version a few days ago, which you can load from the second link at http://dannyg.com/iphone.

22. On Thursday 19 July 2007, 15:18 by Martin R

This sounds like a fantastic breakthrough. Can't wait for it to work, especially for web pages. For me, PDFs aren't as important but also sound great.

# 5 David, you posted the code to make this work. It is not functioning on my iPhone or Mac's Safari. Somehow the spacing is not going into a link right for me. But x=new has a space before the XML and any combo of no space or percent20 space added in the URL if left alone don't seem to do anything for me. More explanation would be fantastic.

23. On Thursday 19 July 2007, 20:14 by Dave Keay

After seeing this post, a friend and I put together some scripts in a really easy to use package - drop a file on it, and you get a bookmark you can sync with your phone!

We called it Filemark Maker, and It's at http://www.insanelygreattees.com/ne...

24. On Friday 20 July 2007, 08:42 by Chris Patterson

The Wikipedia article on data: URIs links to this fine online data: URI generator:

http://www.scalora.org/projects/uri...

Use "HTML" mode and copy the resulting anchor href value or img src value.

25. On Monday 23 July 2007, 08:50 by ab

>> Here is a quick development package, all in the URL ;-)
(see hansp post of 17 July 2007)

This script allowing modify and save for bookmarks
is clever. I could only get it to function
on Firefox, not the iphone. Anyone come up with
a method of editing data url pages on the iphone?

26. On Monday 23 July 2007, 17:11 by Kee Hinckley

Thanks for the inspiration of the one-liner. That led me to waste most of the afternoon completing the task. http://www.somewhere.com/url2iphone... will let you enter the URL of a web site and get back a complete package--images, css and javascript--as a single data URL. I'm sure it's buggy, but feel free to try it out.

27. On Wednesday 8 August 2007, 20:47 by MJ

Sorry, I have very limited programing experience but plenty of alcohol. I just want to play freaking shredder chess offline. Can someone tell me how to do it step by step for someone that is good at following detailed instructions, but not that program savvy? I also am in the process of converting to mac entirely, but still only have a PC at home.

28. On Friday 10 August 2007, 22:01 by iPhone Developer

I have tested the data: URL functionality on the iPhone. If you have read the documentation on the data: URL stuff (see RFC 2397) you will note warnings about length of the <data> stream. I've done some testing tonight and the limit on Safari 3.0 on the iPhone is 128K. This is enough for some fairly sophisticated app development using HTTP, XML, CSS, and Javascript and especially the xmlhttprequest object in JavaScript, but that's about it.

29. On Sunday 4 May 2008, 08:01 by Natasha

I have found two interesting sources ( http://filesfinds.com & http://fileshunt.com ) and would like to give the benefit of my experience to you.

30. On Friday 30 May 2008, 21:37 by Tessa

I am not very great at these types of things. [totally illiterate.] but could someone explain how to do that, in simpler terms that a slow person like me could undertand thanks in advance. :)

31. On Thursday 10 July 2008, 10:29 by curious

sorry for sounding like a total idiot - what is the javascript / php or html equivalent so you can retrieve data from a bookmark ?

$_GET .. something #elitecss ? lets say my bookmark is called "elitecss" which has url address of an entire css file. anyone ? an (detailed) example would be great

32. On Tuesday 15 July 2008, 17:15 by cookie monster

I'm trying to figure out a way to use cookies with this. Any suggestions on how I can set the domain property of a cookie to be any url that starts in "data:" using javascript? That way you could pass data between two different data urls

They posted on the same topic

1. On Monday 16 July 2007, 22:44 by Hackszine.com

Offline iPhone apps with RFC 2397

An old and forgotten RFC provides a way to create applications for the iPhone that people can save and execute locally, even when offline! Back in November, I wrote about how you can store whole images inside of a URL......

This post's comments feed

Life with Lunchhooks