Hello Peeksters,

This last week we've had some growing pains related to capacity. Last Tuesday, our systems ran into some problems and we had to literally shut them off for a couple of hours to clean them up. This was the unfortunate outage that effected everyone -- including me and the whole Peek team. (We all rely on our Peeks 100% and notice every twitch, right along with the Super Peeksters.)

Turns out we are getting pretty big now. Peeksters have been streaming onto the network these last few months and we've been running near our systems' capacity. We've got a heavy duty architecture plan in the works to add tons more capacity, but it's still "coming soon". In the meantime, being near capacity created a really unpleasant aftershock to the outage last Tuesday: lags for many Peeksters in what they were receiving.

Lags. What does that mean?

Some Peeksters are falling behind on their emails by as much as hours. This problem is worse during the day; heavy daytime email flows get our systems even more loaded up. As we have delved into this we have seen some of the culprits -- users who get attachments (especially big attachments) are way more likely to have this issue; another culprit is the overall size of our database (crossing tens of millions of records); another contributor is the mail services that do not provide real time links to Peek -- like Yahoo and AOL. Those services cause much more activity against the database since we have to check them all the time.

So that's the geeky background. The bottom line though is that we are getting busier and busier.

What are we doing about it?

We are fixing it. Some of it is straight problem solving, but some of it is going to really enhance the way the service works for everyone.
1. We are going to add a lot more capacity to the database. This is coming soon.
2. Making the attachment processing more efficient
3. We are moving all Hotmail.com and Live.com accounts to a new real-time interface that we built in collaboration with Microsoft (it has taken ages and ages but it's ready to go!)
4. AOL too -- in a few weeks, AOL will also be on a real-time interface. This and the Hotmail change will reduce load, but it will make the mails for those users come in much faster. Instead of a few minutes before they hit the Peek servers, it will be seconds. Then of course we push out to Peek devices ASAP
5. Yahoo too! That one is going to take more time though. We'll update you there
6. We are going to clean up our databases and prioritize their functions to keep message delivery snappy rather than prioritizing archiving every attachment. This means we'll deliver as fast as possible, even if this means some attachments are not available on your Peek (we won't mess with anything on your PC)

Tonight and the next few nights we are going to make a bunch of enhancements to this effect, and we are expecting that these lags should be 100% gone in the coming days (by June 1) if not already. (As of right now -- May 27 at 630pm EDT -- the system is showing that all lags are cleared. We're going to try to keep it that way!)

Are we getting better overall?

As we have been growing the service, there are other places we occasionally ran into problems that disrupted the service -- the wireless carrier, the email service providers, the configurations on the Peek client software, the network coverage, and lots more stuff. Every time we predict or run into a problem, we fix it. The issues I just mentioned are things we have spent tons of time fixing, monitoring, preventing. We try to learn from the past! Back last fall, we were at about 98% system availaiblity. This year, we have been over 99%. We are aiming for 99.9% next.

In fact, on the device too we are constantly improving the product -- we learned from the bugs in the early Peeks and fixed them in more recent Peeks, we found the glitches in the early couple of hundred Peek Prontos and fixed them in subsequent Prontos.

Here are some principles that we are committed to:
- We want to keep things simple and low cost for Peeksters
- If we make mistakes, we will be honest and tell you about it. That's why you'll hear us say "yes we know about that problem and we are sorry"
- If we can make it up to you -- free upgrades to any Peek device, we'll pay for shipping, if your service was down we won't charge you for it
- We will keep improving the service -- both its reliability and its usefulness to everyone

We're not perfect yet but we're going to keep improving.

Here's a piece of good news that's behind a lot of this: the Peekster community is growing fast! And you are among the first folks to have discovered how a simple and affordable gadget can change people's lives.


Amol

Peekster-in-Chief


PS Email me at asarva at getpeek dot com and I'd be glad to chat with you further. Or just post here...

Thanks, I appreciate the update!

Amol,

Thanks for the update. My personal situation has actually gone from bad to horriffic. I won't bore the group but Dave and I have been trading emails most of the afternoon so he can fill you in if need be.

I'm on the line with Jacqueline at the moment, maybe we'll have a better idea soon.

- Andi

Andi -- total bummer to hear that. I'm listening to updates from Dave and co.

Krauser -- generous of you to offer that! Just stop getting so many darn big emails :) Not very easy to do that so probably better if we just fix things up

By the way guys -- here is a detailed FAQ on how push works and what kind of real-time experience you should expect

http://www.faq.getpeek.com/?cat=11

@ejunan -- we are going to do a mass email explaining all this in a couple more days. Don't want to leave a cliffhanger just now with a "we think it will be fixed soon". But by all means, if you think people want to hear it please say so. Peek HQ is listening to your arguments.

@seanlaf Thanks for the comments. We don't always know what everyone out there is seeing -- the 10 minute thing for example, we thought it was actually the case! Folks commenting and emailing tells us what's happening in the field.

About making it up to you all -- we see the #1 priority to make the service work as promised and then to make it work even better than that. Fast, reliable email; lots of mailhosts; low cost; better texting and more social messaging. And, every enhancement goes to everyone for free. Those are some of the big priorities. If you have other ideas, feel free. If it's a matter of more free credit, please, by all means just email me.

Today was another day of poor service. I read Amol's posting above, and I am very curious, didn't you all at Peek expect that there would be these bottle neck problems once the Peek device became more popular? I am thrilled that your company is growing, and that more people are buying and using the Peek. I do wish it was a public company, because I am convinced it would be a great stock to won, on the other hand, hadn't you planned for the growth? My personal commitment to the Peek is evident in that I bought two spare 1.04's that I am going to send in for upgrade to Pronto. I am merely curious about your strategic planning (this is due to my frustration over the delays. You see, I have become dependent on and rely on my Peek for on-the-go emails)

That email just came through. it was only two hours...

I'm still going anywhere from three to six hours between getting sudden 'batches' of email.

UPDATE: I had something go into my email at 6:35, and it just hit my Peek at 7:10.

ANOTHER UPDATE: email from 7:49 arrived at 8:05; 8:06 mail arrived at 8:14. It's getting better!

dunno bout you Lmaris but looks pretty nice to me: http://www.lakehouse.com/page-68144.html

maybe a bit rough around the edges but seems to me like its still a good buy.

@krauser -- very funny. I was looking for a summer vacation destination just yesterday...

That's a pretty crappy way to promote your blog. If you have an issue voice it here or there, dont provide a link to your blog just to bitch.

Hey Cargo,

Would you mind emailing me at kevin AT getpeek DAWT com. I'd like to see if I can track down why you're still getting one-offs on your account.

Thanks!

What are the current expected latencies for Gmail, pop3, yahoo, etc?

@rayray2012 thanks for the input. It seems varied amongst providers then. I was hoping for an official response. Amol said above that Gmail was under 1 min, but I continue to see up to 20 min. and am trying to tell if my Peek has a problem or things are still wonky.

I'm new to peek and I'm not getting squat (SMS, Email, etc.). I know my email service is working because when I send something to gmail or hotmail it shows up on my windows mobile phone in 10 seconds or so. Peek delays can't be related to the "outage", because that has supposedly been resolved. So, I'm now looking at my second call to support within 24 hours just to get things started. Not a good way to start a relationship.

So, here's what I've tested so far, clearly demonstrating that the Peak Servers cannot communicate effectively with the email servers they are picking up messages from. Seems to me it would be easier for Peak just to drop fully onto the T-Mobile network for SMS and email than to try to create their own infrastructure as an extension of it.

One more week and I'll just go back to my carrier plan; I can't afford these kinds of difficulties or delays in messaging.

----------------------------------------------
ATT SMS to email gateway from cell phone to Peak email address (hotmail)
result: successful; email showed up in hotmail (and blackberry email) in approximately 10 seconds
message showed up in Peak in 11 minutes
reply to ATT phone from Peak: successful, approximately 1 minute

ATT SMS to email gateway from cell phone to Peak email address (gmail)
result: successful; email showed up in gmail (and blackberry email) in approximately 10 seconds
message showed up in Peak in 7 minutes
reply to ATT phone from Peak: successful, approximately 1 minute

Peak to ATT SMS phone number:
result: successful, approximately 10 seconds
reply from ATT phone to Peak: successful approximately 3 seconds

ATT phone SMS to blackberry SMS
result: successful, approximately 3 seconds
reply from BB phone to ATT phone: successful, approximately 3 seconds

Several days in and still no better performance (10 - 15 minutes for message delivery to the peek). I realize this is a new service, but whatever is going on is either clearly not being managed/implemented properly from a technical perspective, or the service is just not going to be able to perform as advertised and we'll be hamstrung until the company is sold and folded into a carrier's product line....

Any news from Peek on what's going on, exactly, and when it's really(!) going to be resolved?

It seems like things have resolved themselves since my last post at least. My hotmail is still around 10 minutes for email but texting to that email are pretty much instant. Its at a relatively acceptable speed and things seem to be getting better.

How fast should I expect gmail to be at this point?

I get confused when they refer to some accounts as "instant". 5 to 15 minutes isn't instant.

Is "push" back online for any email providers? Which provider seems to be the fastest on Peek at the moment?

Ive not received any new mail since 5/29 Ive turned it off hit the read and send Nothing This is a bit long dont you think?

Turned it on and off reset my email and received only two emails today out of about 60 or so in my reg email box This is not good..no mail but they did charge my account for the monthly fee today that seems to be working well.

@lars - When I asked in an email to support about push, I was told "Gmail is about a 7-10 minute delay if you're not using push. At this time, you're not until we migrate your account in the next couple of weeks." - so it seems that maybe some people have push and some do not. 3 follow up emails asking for more information have been ignored.