Gadgets & Gear – Oren Sreebny

Where 2.0 Online – Jeffrey Powers on iPhone vision

Jeffrey Powers, Occipital – Computer Vision and the iPhone Camera

All about getting the iPhone to do the things Iron Man’s helmet can do. REcognize faces and structures, show objects in augmented reality fashion, using the camera. They created RedLaser, a vision-based barcode scanner.

Getting started. First need to interface with the camera. UIImagePickerController – the interface with the camera. Need to check for camera, because iPod Touch doesn’t have one.

Snapture – pinch to zoom in live image capture. Uses CameraViewTransform to do the transform.

RedLaser – can handle blurry barcodes. That matters because the camera doesn’t have autofocus until 3gs. Custom overlay – puts UI on top of the live camera image. Rapid screenshot captrue – currently requires an unpublished function. Image processing – ignores gray scale. Cleans up images. How do you access raw pixels in code? Use DataProvider to read the pixels.

Future of iPhone computer vision – still can’t access video frames. Can’t show objects on top of screen shots for vision-based augmented reality. Mobile computer vision will eventually become a part of most apps we already use today.

Where 2.0 Online – More on iPhone sensors

Derek Smith (SimpleGeo)- augmented reality SDK for the iPhone

3 important technologies – camera, location, and compass. With data from location and compass can plot objects. Can calculate bearing and distance of objects from the device of an object. Device will be at origin of graph. That’s the first stage. The viewport (what the device can see) is the second stage. The third is sizing the objects according to distance. Implemented in OpenGL ES for you. Most of the UI framework doesn’t gel with OpenGL so you have to get creative. If you work in 2D you have to implement your own pipeline, but you can use the standard UI framework.

This was not a very together presentation, but the SDK looks like it will be very interesting when it gets released.

Nicola Radacher – Mobilizy –

example of wikitude client.

GPS signal – accuracy can be bad due to city density or fog. What can you do to improve it? One way is image recognition. Take a picture, send it to a server, compare to data in database, correct user’s location. You need a lot of data for any big city.

What to do if there’s no compass – Calculate position through GPS signal changes. Don’t need user feedback, but it’s inaccurate. Alternatively, ask the user to help – tell them to adjust the phone to point north, for example, or point it to the sun (not good in Seattle!). More accurate than GPS (perhaps), but still not great.

Alok Deshpande (loopt) – CoreLocation in Practice

Nice abstraction built on several technologies. Shields you a little bit from having to worry about which technologies are available. It’s a subscription model. You can specify accuracy and how often you want to be updated. You’re then sent location events with location info. What accuracy do you need? How frequently do you need to be notified of changes? Tradeoff is response time and battery use vs. accuracy. Example: Where’s my car? Simplest way to start is with MapKit framework instead of CoreLocation. Supports showing a user’s location. To do anything more substantial you need to use CoreLocation itself. Probably want to set user’s location to as accurate as possible and continuous update (as they’re walking to the car).

Nick from Skyhook Wireless

CellID, WiFi, and GPS. Skyhook uses WiFi to calculate location. Available on many platforms.

Cell – Universal, 150-700m accuracy, 1-2 sec response, low power.
WiFi – Urban indoor/outdoor, 20-40m accuracy, 1-5 sec time to fix, low power
GPS – Outdoor/ limited indoor, 10m accuracy, 1-60 sec time to fix, medium power

Typical GPS receivers need -140dBm or better. Most cannot decode below -145dBm, or -155dBm with assitance. 140dBm = 10(-14)mW.

WiFi positioning – scan for signals, trilaterate to determine location. in iPhone reports lat/long to CoreLocation

They drive around collecting wifi signal fingerprints then calculate AP position by reverse trilateration.

Martin Roth (Reality Jockey)- Augmented Audio – A new musical world (the mic as sensor)

http://rjdj.me/

What is RjDj? a reactive music player. Reactive music? it changes with your environment and actions.

Uses PureData – visual signal flow programming language to do the input processing.

iPhone has a number of audio frameworks. Media Player gives you access to iphones library. Av Foundation Framework gets you up and running. Audio Toolbox framework plays audio with synchronization capabilities, access streams, convert formats,etc.

Audio Unit framework uses audio processing plugins

OpenAL framework – meant for games.

Where 2.0 Online – Alasdair Allan

I’m participating in O’Reilly’s Where 2.0 Online conference – fall 2009 – the topic is An emphasis on iPhone sensors.

First up is Alasdair Allan – http://www.dailyack.com/ – author of a book on iphone programming. The sensors in your iphone.

GPS (core location) – abstraction layer in front of different methods. Abstracts cell towers (12km falling to 1-3km), Skyhook wireless (approx 100m), GPS (approx. 40m). Have to check if location services are enabled first. iPhone simulator will always report location of 1 Infinite Loop in Cupertino.

Distance filter – can set to update based on distance of a change, so you don’t get so many update messages.

Can set desired accuracy using locationManager.desiredAccuracy

Delegate methods:locationManager gets messages when location changes – new location and old location.

Accelerometer – measures linear acceleration of device – roll and pitch, not yaw (except iphone 3gs if you combine with magnetometer) x=roll, y=pitch. 1.0 = approx 1 gravity. z=front side up or front-side down. 0.0=edge-on to the ground.

Declare view controller class UIAccelerometer instance. Start the accelerometer. Can set update frequency (e.g. .1 sec) – can calculate orientation from that in radians.

Magnetometer (digital compass). Combining heading info (yaw) with roll and pitch, can determine orientation in real time. Only 3gs has this, so important to check whether heading info is available in core loocation with locationManager.headingAvailable.

Magnetometer is not amazingly sensitive – 5 degrees is good for most purposes. Check to see that new heading is >0. Returns magnetic heading, not true. If location services are enabled, then you can also get true heading.

Heavily affected by local magnetic fields.

Camera – you can have the user take a picture and grab it.

Proximity Sensors – turns device’s screen on and off when you make a call – infrared led near earpiece that measures reflection. UIDevice object. Sensor has about a 3.5 cm range.

phonegap is an open source framework for building web apps that become native apps on iPhone and android. http://phonegap.com/

Alasdair recommends the iSumulate app from Vimov.com to be able to simulate acceleromater events in the iPhone SDK – http://vimov.com/isimulate/sdk

Run static analyzer (in xcode in snow leopard) to check your code before shipping to Apple – because they will.

Nice new mini from Dell

I think the Dell Inspiron Mini 9 looks pretty cool. A 9-inch screen, 2.28 lbs, built-in WiFi, and (unlike the MacBook Air), built-in ethernet and two USB ports. $349 gets you a version with Mini OS (Dell’s version of Ubuntu Linux), 512 MB memory, and a 4 GB solid-state drive. Upgrade to 1 GB of … Continue reading “Nice new mini from Dell”

Technorati Tags: dell, hardware

Manually managing music on the iPhone

I’m down in California – had a good meeting yesterday with our colleagues from UC Berkeley about their organizational efforts, the Kuali Student project, and their collaboration tools strategy effort. Great stuff, and great folks! Now I’m in Cupertino for a meeting of the higher-ed iPhone task force. Should be an interesting day. At dinner … Continue reading “Manually managing music on the iPhone”

Now I’m in Cupertino for a meeting of the higher-ed iPhone task force. Should be an interesting day.

At dinner last night I was complaining to Jason Ediger about not being able to manually manage my music on the iPhone by dragging and dropping songs from iTunes. He told me that I was wrong, and that with the iPhone update from January you could actually manually manage music and video on the phone.

And sure enough, he’s right!

If you set your iPhone settings to enable this, which you do like so:

You can then drag and drop songs onto the iPhone – as shown below in this clip:

Cool new goodies from Apple

Watching the blog and chat coverage from Steve Jobs’ Macworld Expo keynote this morning. It’s nice to see Apple introduce some things I’ve been asking for – most notably a light notebook (the MacBook Air – 3 lbs, with lots of nice features including being really thin, a mutlitouch trackpad (like the iPhone), and an … Continue reading “Cool new goodies from Apple”

Watching the blog and chat coverage from Steve Jobs’ Macworld Expo keynote this morning. It’s nice to see Apple introduce some things I’ve been asking for – most notably a light notebook (the MacBook Air – 3 lbs, with lots of nice features including being really thin, a mutlitouch trackpad (like the iPhone), and an (exensive) option for solid state disk instead of spinning disk), and the email app on the iPod Touch (though that’s a $20 software upgrade).

The other thing they introduced that I think is significant is location awareness in the maps app on the iPhone (and the Touch). Interestingly enough, they’re not doing it with GPS (since the devices don’t have GPS, which I’ve heard was a decision based on the power consumption of GPS units), but through triangulation of signal strength from cell or wifi base stations. I know that’s an approach that CS faculty here have also been using in their research projects. I wonder if that location info will be available to applications when they release the iPhone SDK next month.

Not a revolutionary set of new products, but certainly some nice incremental progress from Apple.

Is the iPhone enterprise ready? Sure – why not?

I didn’t listen to the Burton Group briefing on whether or not the iPhone is “enterprise ready” (apparently Burton thinks not), but I agree with Bob Blakeley’s view as expressed by Phil Windley: While you can certainly make a case that encrypting data on the device (even contacts) is necessary for many enterprises, the model … Continue reading “Is the iPhone enterprise ready? Sure – why not?”

While you can certainly make a case that encrypting data on the device (even contacts) is necessary for many enterprises, the model that keeps apps and data on the Web–removing the need for these to be remotely managed–is exactly the kind of mobile platform enterprises ought to want.

There are applications you can think of–field technicians in areas with poor connectivity who need access to large amounts of data–but those are probably the exception, not the rule. Most road warriors could use Web-based tools with little loss in productivity. I have been amazed at the richness of some of the iPhone applications that I’ve seen and it’s only been a few months.

Technorati Tags: apple, iphone, enterprise

Some Macs go missing at the Apple Store

When I was in the Apple Store at University Village on Monday waiting to talk to them about my deadened iMac, I noticed that there was not a single Mac Mini nor a Mac Pro to be seen on the floor, nor any mention of them. Interesting. Technorati Tags: apple

Interesting.

Technorati Tags: apple

Things fall apart – including Macs

The day before I left for vacation the screen went blank on our family room iMac (a 20-inch g4 “desk lamp” model). When we got home on Sunday, Michele impressed on me how important that computer is in the life of our household. The screen had a barely discernible glow to it, so I knew … Continue reading “Things fall apart – including Macs”

The screen had a barely discernible glow to it, so I knew the screen itself wasn’t totally dead, and the computer continued to serve up documents to its locally attached printer from other machines in the house, so the computer was working.

So yesterday I trotted it down to the Apple Store to see what they could figure out. The verdict was that the part of the logic board that drives the internal display was dead. The machine works fine with an external display. That’s a bummer with an all-in-one device that I bought particularly because of the gorgeous 20-inch wide screen.

Of course the machine’s Apple Care extended warranty program expired on June 29. Sigh..

$635 estimated repair bill. Growl.

This boy is not a happy camper. Wail

I’m not about to sink that amount into a three year old computer. More sighs.

I guess I’ll buy a new iMac for the family room and stick the old one downstairs with an external display on it (alongside the five year old Dell that has been chugging along with no problem).

But I’m not happy about having to spend that kind of dough right now.

Technorati Tags: apple, hardware

A good iPhone review with an enterprise slant from Information Week

John Welch has a good review online in Information Week titled Two Weeks With An iPhone that talks about the iPhone with a particular slant of using the iPhone in enterprise environments. It’s worth a read – though I have to admit to being biased to liking the only other person I’ve heard of who … Continue reading “A good iPhone review with an enterprise slant from Information Week”

John Welch has a good review online in Information Week titled Two Weeks With An iPhone that talks about the iPhone with a particular slant of using the iPhone in enterprise environments. It’s worth a read – though I have to admit to being biased to liking the only other person I’ve heard of who admits to having owned a Kyocera 6035 Smartphone (which was, not coincidentally, the last phone before the iPhone I owned that had the sense to have a dedicated vibrate/ring button).

One of the first things he talks about in the review is calendar syncing, and he agrees with what I’ve been guessing at, which is that we’ll see over-the-air calendar synchronization via CalDAV when Apple releases the Leopard version of OS X. He also takes a guess as to what that might mean for Exchange users, who currently can’t sync calendar entries to the iPhone, and he thinks it likely that the iPhone will also do LDAP in that time frame, which would be lovely:

The truth is, until Mac OS X Leopard is released, I doubt that there will be any options for over the air (OTA) sync of anything other than e-mail. Currently, Apple doesn’t have a calendaring solution. They don’t have a really good way to deal with networked user contact databases. Since there’s no provision for OTA sync of contacts and events to any kind of server, third-party support for this is, shall we say, tricky.

However, come October and the release of Leopard Server, that changes. Apple will have a calendaring/group contact solution. I’ll give you 80% odds right now that within a few weeks, if not days of the release of Leopard, you’re going to see an update to the iPhone which will allow for OTA sync to CalDAV servers, and probably some OTA LDAP love, too. After all, why would Apple keep the iPhone from connecting to its own products? I quote from the Chewbacca Defense: “It does not make sense.”

Once you have published ways to get contact and event data in and out of the iPhone over the air, then dealing with Exchange/Domino-style connectivity becomes far simpler, as you only have to make your server act in a way that’s compatible with the iPhone. So I’ll hazard that, post-Leopard, iPhone connectivity will get a lot easier.

We know that Apple is using CalDAV for its calendar client/server protocol in Leopard, and that Oracle will also be using CalDAV for Oracle Calendar (along with others like Novell and Kerio). Hopefully as this new protocol gains adherents we’ll see Microsoft engineer CalDAV functionality into Exchange and Outlook, or at the very least we’re likely to see third-party vendors build add-ons for those products that speak CalDAV. It’s interesting to think that the impact of the iPhone could end up driving the adoption of this new open protocol.

Technorati Tags: apple, calendaring, iphone, mobile-sync, wireless