Angry Robot

RE: Magic Pens: Nebo

Funny how just after I post that article talking about magic notebooks and how I use One Note for handwriting recognition and how it’s not quite there yet, I discover this app Nebo.

Nebo is a handwriting recognition app. And it works. Really well. Just to prove it to you, I’m using it right now. here’s the screenshot of what I wrote:

Part of what makes it effective is that it displays its interpretation of your scrawl live as you write. So if you do get a little too sloppy – ironically, this happened to me with the wards “handwriting recognition” – you can immediately adjust. It has a lot of neat pencil gesture tricks, too. Like to erase a word, you scratch it out. Or underline for bold, etc. But the major part of it is that the text conversion works really, really well. Living-in-the-future well.

Anyway, this app is taking a place in my home screen IMMEDIATELY.

Magic Notebook

I wrote seven years ago about magic books. By that I meant various technologies that were transforming the way I/we read: RSS, read later services like Instapaper, e-readers like the Kindle, audiobooks and podcasts, etc. etc. What would a magic notebook be? Something that transformed the way we create, not just consume? For many years this has been a notebook computer, i.e. a laptop. But for me, the magic notebook is now the iPad Pro.

Pencil

Let me go out on a limb and declare writing with a pen or pencil as far more natural-feeling than typing on a keyboard. We learn to do it earlier, and because it’s easier to switch to drawing, and to use space to arrange your writing, it’s more versatile. I love handwriting, especially for brainstorming. But I’m not a Moleskine dude. I’m terrible at keeping track of physical pieces of paper and I am so totally into having everything all digital and linkable and searchable and easily revised.

I’ve had iPads since the first and almost as early on I was buying various styli. My dream was, I hand write something onto the sheer sparkling glass of the iPad and it converts itself into glorious digital text. Now that’s a magic notebook. But it was not to be, not then. The early styli felt like writing with someone else’s finger. But I knew I wanted to play around with the iPad Pros once they came out, more or less because of the Pencil.

So I got one. (Note: I got one a couple months ago, so I’m referring to the previous generation iPad Pro 9.7”, not the new ones that just came out (d’oh!).) And: my dream isn’t here, but it’s a hell of a lot closer.

The pencil works really well for my purposes, which are short on drawing, long on hand-written notes. I use OneNote and although the Pencil support isn’t the greatest, it lets me keep everything in one place. Also, it has handwriting-to-text! It’s not great, especially as my handwriting looks like something you might write in mud with a stick as death spasms wracked your mutilated body. Plus, seems that feature only works on Microsoft OSes right now (speaking of death spasms).

UPDATE: See this post about the app Nebo for some seriously next-level handwriting recognition.

But it’s getting there. So let’s imagine a near-future utopian world in which Microsoft has enabled this on the iPad. Then let’s imagine that same world where the machine vision craziness has continued to proceed apace. Assuming we’re not all cowering in the muck as T1000s hunt us down to force us into indentured servitude in SkyNet’s underground sugar mills – or something – then perhaps my dream will be here. Writing longhand, tapping convert and leaning back as my insta-posted genius blog post / promo script goes out into the world and somehow causes dollar bills basically to start flying out of my screen as I give a double thumbs up and grin wildly, revealing a full set of solid gold shark teeth…

Sorry, where was I?

Something something iPad something.

The Other Accessories

So anyway, I’m using it for brainstorming and rough notes and then kinda formulating my first draft with an actual keyboard. And oddly enough, here’s where two other Microsoft products make an appearance, in hardware form.

Number one is a $5 loop with adhesive, the Surface Pen Loop, which you can use to affix your Apple Pencil… well, anywhere really, but I’m sticking it to the edge of my case like so:

That way the pencil’s always with the iPad.

The case is a cheapo number, the Moko Ultra Slim. I’m… very fussy. There, I admitted it. So I bought two other cases and returned them, and tried a few others in-store, and this $17 Moko is my favourite. It’s like an Apple Smart Cover in principle, except a lot cheaper and the back is clear plastic. Importantly for me, it’s light and slim.

When I do need a keyboard I have Microsoft product number two, The Microsoft Universal Foldable Keyboard. Like I said I tried a lot of cases, including the two premier keyboard cases, the Apple Smart Keyboard and the Logitech Create. I didn’t find the Apple Keyboard comfortable to type on. The Logitech Create was great for typing, but as a case it’s not great – it doubles the weight of the iPad, for one. And both of them are expensive. The Microsoft folding one is thin and light, costs half as much, and stays in my bag until I need it. Most of the time I don’t need the keyboard, and so the iPad is as light and portable as possible until I do. in that case it folds out wider than the screen of the tablet, giving the keys more room. I will not lie: some of them are still cramped. But I find it quite agreeable to type on. I still have a full-sized Bluetooth keyboard at home and I figure if I ever need to write for hours on my iPad I will bring that.

The Experience

Short answer: I fucking love this thing.

Slightly longer: this iPad is my favourite thing for: reading the news, browsing the web, editing photos, writing, and general purpose computing. Certain things I still need a computer for: video editing, web stuff (although thanks to the incredible Coda I still do a bunch on the iPad), design. The pro apps are still lacking on the iPad. I love the portability compared to a laptop – I mostly have it in my bag by default.

The Future

My iPad is already out of date with the advent of the 10.5″ model, which has a slightly bigger and significantly better screen, is faster, and boasts better pencil performance. Thing is, I’m happy enough with this one to not care. I’m good. Mine is already overkill in all of those areas. (It could be, though, that the Smart Keyboard is greatly improved by the added width; that’s my one regret with this model, but it’s not much of one.)

IOS 11 has a shitload of iPad-specific improvements: easier app switching, better multitasking, a file browser, and more pencil functions. I’m excited about all of those.

The area that needs real improvement is the pro app world. I can’t find a good pro video editor: where is Final Cut iOS? I am having trouble finding a design app that meets my needs. I’ve run into situations with relatively simple photo manipulations, say wanting to rotate a batch of images at once, where the stock photos app doesn’t do the trick and I don’t know where to turn. Hopefully Apple pushes forward in this area. It certainly looks like they are. They’ve been touting the iPad as the future of computing for a while, and only recently have they begun to back that up. And only recently do I think I agree with them.

OCR in OneNote

Here’s a note about notes. It turns out that there is some serious OCR (optical character recognition) in OneNote. I knew text in images and PDFs would show up in searches, which is handy but not mind-blowing as it’s been in Evernote for like 10 years. But what I didn’t know is:

1. You can copy the text ‘out of’ an image and paste it as regular text, or simply convert it. I have been wanting this… For like 10 years.
2. OneNote will OCR hand-written notes. I mean if your hardwiring has fallen into disrepair for lack of use like mine, it will be rocky, but OneNote will give it the college try.

Yup, it’s how I made this note. I wrote it last night with the Apple Pencil on an iPad Pro. I converted it and had to do some editing, and there’s the downside. Of course it’s not 100% or even 80% accurate (again, I mostly blame my handwriting). But also you can’t actually convert to text on the iPad. I’m using the Windows client right now which is the most full-featured, perhaps unsurprisingly. Checking…

No, you can’t do it on the Mac app either. That’s a shame. OCR is working in the background, indexing images so they show up in search results, but there is no feature in the interface that will convert to text.

Anyway, when this feature makes its way into the iOS client, this will be a wonderful option for note-taking. I definitely prefer handwriting for meetings, brainstorming, and some other uses – it’s great just to get away from a keyboard for a bit. It’s certainly one of those things that tablets are good at and feel a lot more delightful than typing. But the ability to turn that scrawl into usable text is to me, kind of a killer app.

The iPad These Days

So the iPad has issues. After exploding out of the gate in 2010, with sales growth greater even than the iPhone, sales have decreased over the past few years, as Horace Dediu discusses in iPad Optics:

The iPad is considered to be failing, with a presumption of an end of life in the near future. The evidence of this failure the year-on-year decline in units sold… The iPad decline is paired with a steady increase in the Mac. The iPad exhibits a four year decrease in overall volumes. This has, as they say, bad optics.

Neil Cybart notes:

A quick look at overall iPad sales reveals an ominous trend. Sales have declined for 12 consecutive quarters. After topping out 74M units in 1Q14, the annualized iPad sales rate has declined by 42% to 43M units.

Yeowch! Except it’s not all gloom. As Dediu notes,

the iPad is still a much loved and much used product. … Tablet ownership among US adults increased from 45% in April 2015 to 48% in April 2016 and 51% in November 2016. The rise has been steady. Although this counts tablets, the iPad had 85% share of the U.S. market for tablets priced above $200 so it’s a fair assumption that the iPad audience is growing.

Furthermore, iPads are still growing in “non-consuming” markets. iPad posted double-digit growth in both Mainland China and India, it continues to attract a very high percentage of first-time tablet buyers.

His explanation for the odd numbers is that “iPads remain in use far longer than phones, and perhaps even longer than some computers.”

Interestingly, Cybart blames iPad’s troubles on something else: its little brother.

People aren’t buying as many iPad mini devices these days. Excluding 7.9-inch iPad mini sales from overall iPad sales results in a completely different sales picture… iPad mini unit sales have declined 70% after peaking in 4Q13 and 1Q14. The product’s value proposition has been permanently reduced due to larger iPhones. Apple has clearly experienced Peak iPad Mini.

Whatever the reasons for the “bad optics” are, Apple is suddenly pushing iPad really hard, as Cybart notes. And they’re pushing it in two different directions, as indicated by the two main model lines. The Pro is more expensive than a normal iPad and has added hardware features, mainly the pencil and the keyboard. The iPad Pros are the subject of a new ad campaign, which Apple is clearly targeting at people who want a PC replacement.

https://www.youtube.com/watch?v=dRM31VRNQw0&feature=youtu.be

Going the other direction, the regular ol’ iPad has just been refreshed. The biggest change is that they dropped the starting price to $329 US. That’s a big deal! The cheap new iPad will compete with Chromebooks in education, it will entice owners of ancient iPads, and it has at least a chance of competing against cheap Android tabs.

There’s a lot interesting about Apple’s strategy on both fronts: they think iPads are their best chance in education, and they think iPads are their best chance at switchers. It wasn’t so long ago it was to the Mac they were encouraging PC users to switch. We’ve known for some time that the iPad was Apple’s “vision for the future of personal computing”. But its latest moves indicate it’s their vision for the present.

iPad Pro?

Some rumours were going around that sounded silly:

[Evercore Partners analyst Patrick] Wang predicts that Apple wouldn’t just simply release a larger iPad — he sees the company using the additional screen real estate to create a hybrid-style device that could serve as both a tablet and a notebook, and would make the iPad lineup more appealing to business customers.

Apple’s never targeted the “enterprise”. (Aside: I know I’m a huge nerd but when I hear that word I always think of the starship rather than business people doing business things). Why would enterprise users need a bigger iPad anyway, and/or why would Apple feel the need to change strategies to compete with Microsoft’s keyboard-wanting Surface, which is not selling anywhere near as well as the iPad? Silliness.

But then I thought about “Pro” in an Apple context, which means creative professionals, and then I realized that Apple’s pro software products – Final Cut and associated apps, Aperture, and Logic – still do not have iOS versions.

In this context, a larger screen iPad would make a lot of sense. I once bought an 11” MacBook Air, hoping to edit a project on the go. It was too small a screen to edit on, however. Certain apps need certain elements on screen at all times. Video editors need a timeline, a clip selection window and a playback window, and when the screen gets too small, the utility of these elements is compromised. Step up to a 13” Air, however, and editing works much better.

It is by no means a given that Apple will release its pro apps on iOS. They may feel that the consumer creative apps – iMovie, iPhoto, and GarageBand – fit the bill. That the iPad is only for playing around, and when you need to do real work, you go to a computer. That doesn’t sound much like this pitch though, does it? Or, you could argue that Apple isn’t that interested in the pro market any more. And as an Aperture user, I can sympathize with that. Surely the pro market is a lower priority than their much larger consumer markets in different categories, but if they didn’t care about pros anymore they wouldn’t have just revamped Final Cut and redesigned the Mac Pro from the ground up.

You could also say that yes, Apple will eventually release Pro apps for iOS but no, that doesn’t mean there will be dedicated Pro iPads. Look at the marketing for the iPhone 5s first, though: “forward thinking”. “the most advanced technology”. “desktop class architecture”. Isn’t that very close to a pitch for pro hardware? Or look at the history of any of any Apple product line since Jobs’ return: start simple, and add an increasing number of models over time. I would expect a bigger, pro iPad, and maybe even a 10” pro iPad. I just wouldn’t bet on when.

Are there other distinctions this thing could have, hardware-wise? Besides the obvious (faster processors, faster graphics, more storage), I would argue for pressure sensitivity. Artists, designers, musicians and even photographers would benefit from it. Would they make a keyboard? I doubt it. When you think of these sorts of jobs – music, design, illustration, film & video, photography – a keyboard is not the tool that they need. If anything, different custom inputs depending on the role might be interesting; imagine a mixing board for music production, for example. But really the touch screen is the ultimate custom input, and is much more direct.

A 13” display isn’t the biggest thing in the world, but if the iPad Pro displays get any bigger – and one would guess that they might eventually – Apple would probably want to make a stand for the thing that would optimize its use on a desk. You would want to approximate the ergonomics of a drafting table; the stand might elevate the thing at a 15° angle and prevent it from sliding around.

The biggest thing holding back such a device isn’t really the hardware at all but rather the software. Apple has just last year completed the consumer-level iLife suite for iOS (do they even call it iLife anymore?). The pro apps would have to be a big upgrade from those. The OS itself may need substantial upgrades as well.

That’s it – some completely unqualified speculatin’ about an unreleased Apple product. Just what the internet needed.