just another sign?

that the present global economic expansion is hitting it's peak and is about to head south again? Seems the signs keep stacking up. Whether it's the likes of Cisco or Dell, or Europe in general or the slow downs over in Asia and perhaps even Brazil ?

20 times as much bandwidth?

full duplex 2Gbps per server? Doesn't seem like much - Is MS implying that the average data center server sits on a 100mbps half duplex connection ? That doesn't sound right. Maybe they are referring to a massively oversubscribed network like maybe 48x1GbE uplinked using a single 10GbE cable or some strange sort of setup like that.

Even going back a couple of years it seemed not outside the realm of possibility that people would have 10GbE all the way down to the server in some hadoop clusters.

Given a small cluster size of 250 nodes it's trivial to provide non blocking bandwidth to that number of systems. Non blocking bandwidth to a few thousand nodes though is a bit more complicated, though not much with today's switching technology.

google abandoned map reduce

news at 11

http://www.theregister.co.uk/2010/09/09/google_caffeine_explained/

citrix

I suppose you could also mention that Citrix uses FreeBSD for their Netscaler load balancer platform (of which they have virtual appliances named VPX).

Though it looks like VPX has been supported on Hyper-V for a couple of years now ?

http://www.citrix.com/English/ne/news/news.asp?newsID=2300624

If that's the case it implies FreeBSD has worked fine for a while, wonder what the new code was about - perhaps performance related or something, maybe native drivers for Hyper-V.

weak portfolio outside of FCoE?

I keep track of what Brocade does from time to time going back to the Foundry days, was just browsing their products again to see what they offer at the moment and can't help but think their 10GbE offering outside of the "converged" lineup seems quite weak, really no products for that market for the past couple years at least. I am told the VDX line up carries a significant price premium over other 10GbE(assuming others typically are not "converged"), so people are probably not going to buy into VDX unless this pricing strategy has changed or unless they are confident they are going to need the converged stuff.

On the Chassis side the have the FastIron SX and BigIron RX seem to be older technology and not as competitive as they should be. Foundry(from what I recall) always seemed to release innovative platforms for quite a while, has Brocade taken too much resources away from non converged ethernet and put them on the VDX line up? Since their non converged offerings seem to have suffered since the converged stuff came about.

Also it seems that their VDX line up still does not support layer 3. I asked one network engineer last year who was proposing VDX for my company - how do I do layer 3, and he said you'll have to get a NetIron to do that. I went to another manufacturer instead. Foundry seemed to me at one time to really never release a product w/o layer 3 support. I'm not talking about needing to run a routing protocol internally, this is simply routing between VLANs, something every switch should be able to do (short of the really dirt cheap low cost ones of course). Also provide a feature like VRRP or something at least.

Brocade's entry into the 10GbE space on the small end was the TurboIron 24X, another product which lacked layer 3 support from the get go, they eventually added it but like some of the other products it's at least two generations behind the competition. If I recall right it took more than two years to get layer 3 on TurboIron. And VDX has said Layer 3 is coming in a "future release" - how hard can it be? Foundry has more than a decade of experience doing Layer 3 on switches.

They recently released some innovative low cost 1GbE options, but I guess that wasn't enough.

what are those 800 VMs doing ?

I looked at the I/O profile of my little VM environment (~150 VMs) which hosts a production and pre production e-commerce site(LAMP), and for the production side average IOPS is 5 IOPS/VM for the VMFS file stores (databases use RDM). Non production side which is a 3rd of the VMs in the cluster at this point in time average is 2.2 IOPS/VM (both numbers average over 30 days). In my experience this is fairly typical per VM I/O(going back across about a dozen clusters running web site infrastructure services over the years). Bulk unstructured data would be housed on a dedicated NAS cluster of sorts(and accessed via in-guest NFS), and structured data goes to RDMs(with I/O measured separately - not included in above numbers). As a result per-VM I/O remains low - basically log files and stuff, with more I/O intensive things on RDMs to leverage storage-based snapshot technology (primarily - also makes it easier to manage space and monitor I/O).

So with those numbers 800 VMs is not hard to accomplish on small storage. Though we chose bigger/better storage primarily for the availability and maturity of the platform. The array itself has been shipping longer than Tintri has been in existence. If we were a bigger company and had more side projects to test stuff out on it would be interesting to see Tintri and others in practice.

I suppose if we built things to run EVERYTHING inside vmware data stores tintri may be advantageous there but I really don't see myself ever doing that, any more than I would see myself building physical servers that ran everything off internal disk drives. For one thing the vmware-based snapshots really are quite limited(and slow!) - I take a vmware snapshot on average maybe once a month. I also like maintaining compatibility with physical infrastructure - knowing I can spin up a physical server, swing an RDM over onto it and keep going. Or share a NAS mount point between a VM and a physical box. I'm not in the camp(and never have been) that believes everything must be a VM.

Production VMFS workload is 96% write, non production workload is 87% write for these particular VMs.

XIV really?

XIV really doesn't compete with anyone at the moment does it? I mean it taps out at under 200 disks. Unless your talking about some sort of XIV+SVC combination.

I've drilled 3PAR over the past 3 years as to why not support more usable capacity, their response is there isn't a market demand for it, the average 3PAR system in the field has about 400TB of raw (or usable I forget, I assume raw) capacity on it. I guess people don't load up their arrays. When I got my T400 back in 2008 I immediately had the first two nodes maxed out for capacity(150TB raw - but not maxed out for I/O), so was always curious why they didn't push capacity further. I guess if the customers aren't pushing for it then there isn't a need to develop it.

As to EMC - don't you find it curious that they haven't gone beyond 8 engines at this point? I mean going back to when they introduced it they were saying how they were going to have many more engines. But for some reason even on this new big 40k they stuck with 8? I'm certain, especially when loaded up with SSD that a VMAX would need far more than 8 engines to really drive SSD performance, since your easily talking about 10s of millions of IOPS potentially anyways.

3PAR has stated they don't intend to go beyond 8 controllers for the foreseeable future, instead on the V-class they doubled up on the ASICs to about double node performance and massively increased memory capacity.

Certainly feels like EMC went back to the drawing board and reconsidered their original plans for a massively scaled out VMAX.

HDS's SPC-1 performance on the VSP was pretty disappointing (though SPC-2 was very impressive), I wonder if this new 40k VMAX will make EMC confident enough to post numbers on that platform, I'm not holding my breath though.

Question remains do those 8 engines have enough processing capacity to drive the full I/O of the back end spindles ? For VSP it seems the answer is no, at least for random I/O. Wouldn't surprise me at all of VMAX was in the same boat.

Wake me up when EMC gets distributed sub disk raid, with the massive nearline drives coming in the future, traditional whole-disk RAID is going to lose out pretty fast with long rebuild times and higher risk of double/triple/more disk failures.

But I'll give them some credit for at least upgrading the CPUs they were using, bout time. Though I'm not up to speed on the latest Intel code words, it seems Westmere is already 2 years old. Day late, dollar short? Oracle at least is pushing quad socket 10-core Intel procs on their systems. Get with the times EMC.

up late to do a software upgrade on one of my 3par boxes..time to go to sleep.

do they have pricing based on utilization?

I've been told it has rarely been used but HP has a model where you get X amount of storage and you pay for what is written, which sounds like a good model for service providers. You can leverage wide striping of say the 3PAR platform (or VMAX if EMC has a similar pricing model available) for good performance from day 1 and are able to spread the costs out as you add in customers. vs a more traditional approach where you may not be able to afford to buy one of those $1M arrays up front and instead start out really small - and the I/O penalty you pay for being small and have a more difficult time building up.

Per the HP program I don't know the details around it, there may be minimum level commitments over time and stuff. The program might of even have existed in 3PAR prior to being bought by HP I don't know, I first heard about it last fall.

3PAR of course has had self provisioning via the Virtual domain software package for years now (up to 1,024 management domains in an array) - though I don't believe there is any specific integration with fancy APIs or metering stuff specific to service providers.

minimal amount of cpu

just give them a VM and crank the CPU mhz limit down. That's what virtualization is for right?

I got tired of waiting but I tried cranking down a CentOS 6 64-bit VM to 66Mhz and started it up from cold boot. after 5 minutes I stopped waiting (it just entered run level 3)

Netapp scale out is not scale out

their "cluster" is little more than a hack it seems like. I drilled what I think was a NetApp employee on some of the finer points of their cluster (given I have not used it, and there seems to be a lot of hype about the release) and was kind of surprised and dissapointed by the results --

http://datacenterdude.com/netapp/netapp-dataontap-81-reponse/

I don't know about Isilon's performance, perhaps they are 'slow' in IOPS but it really seems the system is built for throughput rather than IOPS. I agree it doesn't seem like an ideal platform to run Vmware directly on top of. I assume (hope) that EMC didn't buy Isilon for that market segment. It makes a lot more sense to have Isilon as a scale out NAS where you put your data (directly accessing it via guest OS-based NFS) vs your VM images which you put on more traditional storage like a VNX or V-MAX or whatever. The amount of data for the images usually pales in comparison to the amount of data that the applications are using by orders of magnitude.

The impression I get, is that NetApp continues to bolt stuff onto their system which just makes it more complicated, instead of really addressing the core issues of scalability and even scale out. They probably just have too much invested in the current system to be able to really, truly fix it (much like Cisco). But it seems NetApp is still years away from having what most would consider a real cluster, if they ever get there.

Now how they are able to market the thing and get customers to buy into it is another matter. It wouldn't surprise me if they can sell a few more systems with this, but from a purely technical point of view, as a cluster - NetApp isn't there yet.

Look no further than the lack of ability to stripe a volume across more than a single controller node, even in cluster mode, I mean come on. Take NetApp's 24-node SpecSFS results that they released around the time 8.1 came out. Your basically having to MANUALLY manage 48 different storage systems (because a volume can live on only one controller). If you have a perfectly optimized workload like SpecSFS you can distribute your data over everything, but if your a more traditional user I can imagine it keeping the administrator up at night (unless they are massively over provisioned) because the system can't even automatically move a volume to another controller in the event I/O goes up. And even if it did (Compellent has this ability) - there is quite a large overhead in moving TBs of data around between systems. Instead of having it balanced from the get go like a real cluster should be, and being able to move more finer units of storage around - e.g. sub LUN auto tiering between arrays.

And as you might expect, that data management stuff you speak of (of which I bet part is deduplication) doesn't apply cross cluster nodes either, so are you now going to try to optimize de-duplication by trying to move volumes with like data on subsets of your cluster? Manually?

For all of the hype this release seems to have, if I was a customer I would feel let down - this is all they get after so many years of trying to integrate that Spinmaker(?) stuff ?

Also, if Xtreme IO is built from the ground up to be flash based, I don't see why it would compete with a hybrid NetApp PAM/HDD system. Unless EMC wants to try to integrate the XtremeIO into their existing line up, vs keeping it as a stand alone product(would that take them many years to do like it took NetApp to do with Spinmaker?). Perhaps the acquisition was to fend off the likes of Violin (maybe Xtreme IO came at a much better price)? I don't know. I'm not too familiar with that market space.

I just really don't see from a technical stand point how an Xtreme IO vs NetApp stacks up, they appear to be two completely different approaches to solving different problems. Though that won't stop sales people from using even more force to fit square pegs in round holes.

no resource pooling?

What? no resource pooling? in 2012 ? To put it bluntly - how lame. One of amazon's biggest failings is lack of resource pooling, and HP is going that route too? All of those fixed sizes for VMs is quite archaic as well(same goes to Amazon - fixed instance sizes are reminiscent of the 90s and physical servers!), should be able to specify whatever value I want for cpu/ram/disk, and be able to change them whenever I want. Bonus points if you can both hot add *and* remove memory and cpu and disk from the VMs w/o application impact.

Hopefully HP can provide at least better support vs Amazon, and a better service(which sucks at both when it comes to IaaS), but am not holding my breath.

i agre with paul

stupid idea ... build up higher rather than wider. e.g. shipping containers with ~57U racks.

Myself I love big racks (both kinds). I have grown quite fond of Rittal's 47U, 40-42" deep, but most importantly is the 32" wide (still 19" on the rails), makes life easier for cabling and PDUs. Most facilities today you run out of power long before you run out of rack space so consuming a few more inches of floor space on either side (8" total vs a typical rack which I believe is 24" wide) doesn't really cost you anything, and gets you tons of space for cables. I'm sure other manufacturers have similar racks, Rittal is pretty common though.

I was very impressed with Microsoft's container/rack design when it came out last year, that is quite impressive, seems leagues ahead of facebook's first generation design (not sure if they've revamped it much since the original release).

what's the big deal?

What's the big deal if someone else can find your ip? I find it very humorous that people go out of their way to obscure things like mac addresses, host names, and even IPs (especially internal IPs). Sad too.

no playbook?

I would of thought given the playbook was based on QNX that it wouldn't be difficult to get the new OS up and running on it at least even w/o any phones

more than 10TB

from the looks of it FIO's octal tops out at 10TB for a single card. The cost of such a card makes it worth while to have a server that has a lot of horsepower, say something like a quad socket system that has, nearly a dozen PCIe slots, e.g. DL58G7 which could get 4x10TB cards(PCIe 16x), and a few more 1-2TB cards (PCIe 8x) cards in it at the same time.

Or go bigger with an 8-way DL980 -

http://www.fusionio.com/blog/-fusion-io-powers-new-hp-data-accelerator-solution-for-oracle/

I would think the distinction of the fusion method is you can use pretty much any server hardware you want - even blades (in HP's case anyways since they have FIO modules for their blades), and don't need to wait for the fully integrated stacks to emerge, get the added speed now (assuming you need it - I sure as hell don't!)

Also I'd assume it's easier to get up and going with Fusion - since you can start small(160GB), and grow more later.

Though fusion certainly has a lot more competition now than a few years ago!

more like a server with storage

rather than storage with servers. Especially since it's an appliance.

certainly is a reason to put in memory on a network link

flexibility.

In memory data even over a network link is still much faster (and probably "fast enough" for the vast majority of workloads out there - same goes for SSD over a network link). The main drawback for SSD in traditional enterprise arrays is the cost is really high and the controllers aren't prepared to scale that well with them. If you look at some of the biggest baddest enterprise arrays being able to squeeze out 200-500k IOPS with tons of CPUs, cache etc. When compared to what SSDs can do on their own, there is a large mismatch of scale.

Not only that but take 3PAR since I know that technology very well, the P10000 specs (which is rated for 450k SPC-1 IOPS) say max of 8 SSDs per disk enclosure. So it's not as if you can take a disk enclosure and slam it full of 40 SSDs and off you go - if you want those 40 SSDs your talking about 5 x 4U disk enclosures - 20U of space mind you - half a rack, and at least in the past 3PAR has been anal about their power configuration, if they still are in this scenario your looking at 2x208V 30A circuits to drive those 40 SSDs. Of course they won't draw that - I'm quite certain in fact that the disk chassis themselves will draw significantly more power than the SSDs that they house(especially given there is only 8 per chassis). But most customers pay for power by the circuit rather than utilization. And guess what - you can "only" get 12 disk shelves on a pair of controllers on the P10000(it is, after all 24 FC links to connect 12 chassis). Of course you can put SATA or FC drives in those chassis along side the SSDs, just using this as an example of a SSD-only type of solution to see how crazy it can be.

So it's pretty easy to see why large scale SSD use in a typical enterprise storage array is not nearly as cost effective as something designed from the ground up to be SSD like say a Violin box. Though from what I have heard is Violin lacks a lot of software, it's mostly just a dumb fast storage system, which I'm sure works well for certain use cases too.

But don't think that it's a stupid idea to keep things like in memory and SSDs from being networked just because of latency. Networks add very little latency - especially when your comparing IOPS of memory and SSD to traditional spinning rust(which is often traveling over a very similar network anyways).

mercury

the spec sheet for the HUS seems to indicate that for the first two tiers they are using the BlueArc Mercury 50/55 and on the higher tier the Mercury 100/110.

The throughput of the Mercury seems to lack behind the throughput of the HUS itself. - around 1.4GB/sec peak for a 2-node 50/55 and 4.5GB/sec for a 4-node 100/110 vs between 6GB and 12GB/sec for the HUS back end. Though I suppose if it was an issue HDS might happily switch out the Mercury platform for a Titan platform. Though maybe they are reserving the Titan for their VSP version of the HUS and force customers to go VSP if they want HUS with Titan..

Still don't see any evidence the BlueArc platform supports any sort of thin provisioning, so sharing storage between file vs block you'll probably have to be more careful vs something that could reclaim storage between the different types of access. Too bad the low end doesn't support wide striping? What an odd decision.

Seems to be a really nice upgrade vs the AMS 2k which was a pretty bland system. Given HDS has to have a completely separate "module" for the file access (as opposed to running another software module on the same CPUs like Netapp - and maybe EMC ? does) - I wonder how that will impact the pricing vs the competition.

just a bundle?

So is this HUS just a bundle of an AMS and a bluearc ? Are these HUS back end systems identical to the various AMS2x systems or are they newer/faster/better?

Seems kind of strange to say this but it seems BlueArc still does not support thin provisioning ? I ran a search and it seems the only mention of TP is when using vSphere (and leveraging vSphere's TP), so my question would be to clarify at least thin provisioning integration between the NAS and the SAN and that you could do space reclamation and stuff between the two. It sounds like this won't be possible though.

The BlueArc titan seems overdue for a hardware refresh - the current generation of gear is four years old now. - Mercury is a bit newer though of course a lower tier.

i guess I'm old

I had never heard of the name instagram until it was bought out.

Aren't there plenty of photo sharing things? I recall hearing about various twitter photo sharing things over the past couple of years, then there is things like flickr and others I'm sure that I've never visited or heard of either. Facebook too I hear is popular for sharing photos - I don't use that site either of course.

How much of a market share do they have? From what I have seen so far they started out as an iOS only app (did they have a traditional PC-web browser interface that people used a lot or is the bulk of their stuff mobile only?), and only recently introduced android support?

Doesn't seem to me to be any anti trust things here myself. Don't really see why anyone would care really.

Funny to see financial folks continuing to question how any of these mobile companies are going to make money. They're pretty much all riding on hype, ala dot com days. Even google admits the revenue they get on mobile is tiny - and revenue is difficult to be had because the screens are so small ? less spots to put ads. Tons of companies in the same boat - lots of eyeballs, and traffic but no money. I saw a discussion recently with some folks talking about twitter - suggesting they should "find a stupid company to acquire them" rather than try to go IPO because they haven't demonstrated the ability to make money (at least in those people's eyes, I haven't kept track - I don't use twitter either).

SPC-2 too

someone pointed out to me new results for the SPC-2 for this oracle array, at least from a price/performance perspective the results are impressive

http://www.storageperformance.org/benchmark_results_files/SPC-2/Oracle_SPC-2/B00058_Oracle_ZFS-7420/b00058_Oracle_Sun-ZFS_7420_SPC2_executive-summary.pdf

I'm sure part of the reason is this Oracle 7240 is using 10-core CPUs, a total of 8 of them or 80 CPU cores in their cluster, vs NetApp having only 16 cores.

Another big part of course is they have 6 times the amount of memory+flash vs NetApp (on the SpecSFS test - the SPC-2 test had no flash)

Take those combined and the cost - it's pretty impressive indeed.

Fusion IO partners

also just wanted to point out the obvious that fusion io isn't in the market to compete directly against the likes of HP and IBM - I'm sure they very much want to OEM the technology to those folks, the big three are already big partners of Fusion IO. It wouldn't surprise me at all that at least for a while these guys just certify and re-sell the fusion io tech to their customers. I mean Fusion IO has only one product right - the flash. They don't make servers, don't make arrays, none of that stuff.

HP hasn't made disk drives in ages(I think they used to a LOOONG time ago? I could be mis remembering though), IBM hasn't either. They just go out and acquire the stuff they need and slap their management interfaces on top of it and throw their support behind it and jack up the price.

I'm sure there will be some effort to standardize on the technology so you can take a HP server and connect it to an IBM storage array or Dell server and connect to HP storage and have it be compatible and supported. Those big vendors don't like to lose out on opportunities to get their foot in the door of the competition - and being compatible really helps make it that much easier.

what about these new flash hybrid startups?

I mean if this process that co-ordinates data movement between server-based flash and storage array based flash in an efficient and reliable manor - does that take the gas out of some of these new flash startups?

I suppose it depends on how long it takes for this server flash+storage array co-ordination stuff to take, it sounds pretty complicated.

wake me up

wake me up when amazon

doesn't require you to build your application to fail (that is, not react badly when infrastructure goes bad and doesn't recover),

and when they allow you to pool cpu/memory/io resources,

and when they have a support team that gives a shit about the customer,

and the ability to move VMs off of degraded hosts w/o requiring user interaction,

and when they allow you to reserve a static ip both internally and externally for a VM.

And when their RDS service allows users to skip mysql replication errors (no SUPER rights)

And when their EC2 intsances allow more than one external and/or internal IP to be assigned to it (load balancing purposes - e.g. run the Zeus EC2 package)

And when ELB doesn't blow hard donkey balls

And when EBS latency SLA drops to normal levels (not 200ms as it is now)

and when they integrate things like replication between their regions (as-is, did you know if you have S3 data in region "A" and want to get it to region "B" you have to do it yourself? You can't even establish replication using their own RDS DB system between regions - and you can't replicate from RDS to a non RDS instance either).

did you know that amazon requires you define an "outage window" for all of your EC2 and RDS resources (maybe others too haven't checked?). They reserve the right to take outages on their stuff during those windows and you can't opt out.

I could go on and on and on

it's a joke, really.

(Amazon user off and on for almost 2 years, it's been the worst experience in my professional career and they top it off with staff and support that doesn't care "it's not our fault you didn't build your application right"). But at least I wi ll be rid of amazon soon, last bit of stuff being moved out next month.

curious arista and not force10 - and lack of scale up?

Is the storage hardware based on dell too? Given force10 is owned by Dell they probably would of gotten a good deal to be all Dell. Though maybe Force10 lacks the APIs or something.

Couldn't see whether or not the hardware was swappable, the system seems like it could be constrained by the choice of server hardware with the max VM size being the size of one server or 4 cores and 32GB of ram. For many workloads this is fine but databases and stuff often want or need more. Technically I'm sure it's easy to do just no indication whether or not it's possible in the canned integrated solution.

No obvious indication if the system supports HA - I assume it does. things like moving VMs between servers without impact is pretty important, and something that Amazon can't do.

The choice of Nexenta for storage seems good from a cost perspective but as a Nexenta customer(NAS only - very small installation w/SAN back end) I don't think it would be my first (or second, or third) choice as primary storage for a cloud system. One of my co-workers used to work for a Nexenta reseller and has similar thoughts. The system just needs more work.

But then again I would not be at a company that participates in the great race to the bottom for pricing. Those in the race have few options available to them.

Is the system really 100% SSD? The web site seems to make it seem so. I know Nexenta is agnostic to SSD vs spinning rust so technically there is no limitation, just wondering about the integrated offering.

Seems like a nice approach just needs a few more tiers of capacity - bigger servers, fatter storage (especially with the Nexenta tiering/caching - not that I've used that feature nor do I plan/need to ).

Competing with Amazon on pricing with your own stuff is simple as can be for the most part, the critical failing of the Amazon infrastructure is they lack the ability to pool resources and over subscribe your services. Instead they want you to start and stop VMs on demand and have your resource requirements precisely nailed down - guess what, in most cases that is not possible. You can provide all of the infrastructure APIs in the world to make it easy but there's the little complicated part called the application running on top of the infrastructure and it's dependencies and configuration. Most are not built for that and will continue to not be built for that because only a tiny fraction of customers need it and it's really complicated to get right.

Where Amazon and other players get you is selling you on the 'cloud' concept in itself. Which sounds nice - it's the implementation that sucks. Most people looking at this stuff aren't technical enough to know what is going on and only sometimes do they come to a realization that they are way over spending for something that doesn't work as well as leveraging technology you can get in house to do the same thing. You may not have a self service portal and stuff out of the box in many solutions - though most organizations don't need a self service portal. Unless your big enough to have complicated budgets and charge backs between departments and stuff self service portal drives resource utilization down, and costs up because people are under the impression that the resource is unlimited they can just go provision whatever they want.

An extreme example at one company I was at there was more than 200 VMs in EC2 that was costing over $80k/mo that nobody knew what they did. So I killed them all, nobody noticed. There was high turnover in the company and a mentality that the resource was basically unlimited so they spent massive amounts and got terrible results. Management was convinced cloud was the way to go despite wasteage like that. If you have 200 VMs in house not doing anything, resource utilization on them will be much lower, slashing costs to run those VMs. Because resources are shared they aren't taking away much from the pool. In Amazon each of those VMs has a fixed amount of resources.

Good controls are very important to properly controlling costs and maximizing utilization.

try getting 400GB of ram in a workstation?

ever try that? Yeah, didn't think so. Also having it be non volatile helps since you can turn your computer off at night, or suffer a crash and recover quicker. Staging 400GB of data from disk is going to take a while. The cost of $2500 seems really cheap if that is for 400GB of flash, a fraction of what most of their other products cost (at last check anyways).

As for Cisco becoming one of the largest Fusion IO customers - why would they be ? I mean Dell, HP and IBM have been customers/OEMs of Fusion IO for some time - any idea where they rank on the customer list? HP, to-date, as far as I know is the only one to have OEM'd fusion IO to put on a mez card on their blades.

Obviously HP/Dell/IBM have bigger market shares in servers than Cisco..

If Dell/IBM/HP are each already more than 10% then forget this comment !

10GbaseT?

Might this be the first system to have 10GbaseT as standard ? 4 ports of it even?

if so that's more newsworthy than the rest of the announcement :)

expectations too high

to expect any product to reach a very mature solid state to where a big company would use it in serious production in only two years is expecting too much I think. Especially when it comes to storage(that object storage thing they have) - gotta make sure it's solid.

I suspect the only serious folks using these things are going to have on staff developers that can help maintain/fix issues in the product for some years to come.

WD at 2TB

for the enterprise at least, was looking at them this weekend since I have several of their RE4 2TB disks for my home servers.

Didn't WD just buy Hitachi anyways ? or in the process of buying?

never seen sizing be reasonably effective

I have never - ever seen sizing be reasonably effective, well I take that back I remember one application I supported that had a very basic (but very intense) workload and when we refreshed the servers we did a bunch of testing under high load to see where we could get. But even with that testing we couldn't get more than about 70% cpu usage out of the boxes (8 cores total at the time about 3 years ago) because of thread contention in the app, though it was impressive to see upwards of 3,000 requests a second being processed in a sustained manor, had not seen that level of performance before. That company had another app that wouldn't scale beyond one CPU core. So it was easier to put ESXi on some of our new hosts and run 8 copies of it (far simpler than figuring out a good way to run 8 copies on the same OS instance). Not scaling beyond 1 CPU core was not a big deal when the app first came out in the days of dual socket single core, but when we bumped to 8 cores well it was more of a sore point.

Anyways back to original point - short of that one app - I've never seen hardware sized effectively - in the vast VAST majority of cases the cpu sits idle at 10-20% with maybe spikes to 30% whether it is physical or virtual - even the CPUs of VM hosts sit idle at low levels! It's all about memory capacity (not even memory performance), cramming more of these low usage apps on the same systems - so I want more cores to run more VMs.

One developer asked me yesterday how heavily loaded one of our apps was on our VM infrastructure he was going to make a change and wasn't sure if it was going to cause them harm. I looked we have 2 VMs for each of these apps, and the VMs at the moment have 4 vCPUs on them ("just in case" - our new VM cluster is still new and we're still measuring the performance after moving out of Amazon). So I looked - guess what ? The average CPU usage on those 4 VMs was 0.25% -- not even 1%. There was some spikes to 10% when some batch jobs run but otherwise 1% cpu.

Every org I've ever worked for has never had any idea on how to size things so frequently things are sized to budget / power / rack space rather than the app. I'm not going to go out and buy a bunch of servers that have 2 CPU cores each and 8GB of ram because that's all the app can use it's a waste of power. Run more apps on the same boxes(one company I had 10+ different ruby apps running in different apache instances on the same small collection of servers).

One company I was at about a decade ago now, it was not uncommon for us to be forced to double server capacity literally over night after a major software release because the software was so bad (and they had no time and ability to perform adequate performance testing). The server counts in the early days were small. That same company spent a bunch of cash for their Itanium-based Oracle HPUX servers, only to have them hover at 80% I/O wait

Having worked for companies that develop and operate the own in house software and run it ourselves I suppose puts me in a more unique position than a company running off the shelf stuff with a fixed number of users.

Then there were the days of fighting Oracle latch contention where the experts say you can throw all the hardware at it that you can buy and it won't fix the problem. Gotta fix the queries.

The lack of resource pooling at all levels whether it is storage, memory or cpu is the most critical failing of the public clouds (especially Amazon). Some enterprise cloud players offer resource pooling but it seems limited to their enterprise agreements not on demand stuff.

At least with virtualization if you make a capacity mistake it's a lot easier to correct (up to a point at least). Which is one of the main things I love about 3PAR - I can change the data distribution on the back end at any time, I can make a mistake and fix it later without impacting the applications in any way.

I've grown more attached to the older term of 'utility computing' rather than 'cloud'. At least when it comes to infrastructure. True cloud, from a provider standpoint is very, VERY complicated. Utility computing on the other hand is really simple. I think people think too much about cloud when for the most part utility is all they need.

10 hours to 10 minutes?

One of my friends at Vertica said their product (w/o tuning and on lesser hardware than the customer had on their production system) took one of their most expensive reporting queries from 24 hours to 9 seconds.

I'm sure most will say hard to believe or not possible but it's pretty amazing.

had to mention it since the article mentioned 'reporting'.

I came across the MS stuff on XIO's website recently it for sure sounds impressive, they seem to be getting close to Redmond, I noticed their volume/node spanning feature only supports windows.

http://xiostorage.com/products/x-volume/

Still kind of disappointed as to the lack of port density on those XIO boxes, only FC, no iSCSI and only 2 ports(if I remember right). I certainly like FC more than iSCSI but iSCSI does offer some nice flexibility, so it's nice to have both available.

only 340MB?

seems like that would fit pretty easily in the RAM cache..

hopefully vertica works out better

than arrowpoint or that acopia product. I remember F5 trying to push that onto me after they acquired it, at one point they were pushing to get just *one* customer (I have several friends internal to F5, had more at the time than I do now some have left). It just doesn't (and still doesn't) make sense for F5 to have such a product. But they had(probably still have) plenty of cash to throw around.

vertica is a pretty amazing product, I am by no means an analytics guy, I was introduced to Vertica about a year ago in combination with Tableau software and left my jaw on the floor at the end of the presentation, not many products do that for me anyways. The sheer performance is just amazing.

One of my friends is a sales rep at Vertica and he tells me time and time again the product more or less sells itself (also a rarity in almost any IT market in my experience), the performance gains (even before optimizing the vertica cluster) are just unbelievable until you see it first hand. And they make it easy to try (unlike much of their direct competition) by making it downloadable and supportable on physical hardware, virtual hardware, and even within Amazon EC2 (ugh, but nice that they give customers the option).

Tableau on it's own is an amazing product, but it's true power can't be seen without something like vertica behind it, with something like MySQL you'll be waiting...and waiting...for queries to return. One developer I worked with last year was too impatient for me to get round to setting up a vertica cluster and he pointed Tableau at an existing MySQL reporting host, he gave up on it after waiting 15 minutes for the first query to return from MySQL.

others will jump in too

I'm sure if enterprise use of this cloud storage starts to take off the likes of HP, NetApp, EMC etc will respond with products that can integrate there, at least at the file/NAS level, integrating the cloud tier into the box itself.

S3 is one service I have to admit Amazon does pretty well for what it provides. The rest of their cloud is absolute shit, but S3 isn't so bad. Still latency and bandwidth are still the biggest factors here, even with a local cache how much cache are you going to have? Users are not going to want to wait 20 minutes while an appliance downloads something from the cloud, and companies are probably going to find massive internet pipes to access cloud storage is probably going to cost them more than having storage locally. The amount of data being stored is rising at a much much faster pace than the decline of the cost of bandwidth. For me with a gigabit connection with about 17ms of latency to S3 I manage to get about 4MB/second of throughput from them.

I can see cloud storage (in conjunction with encryption) replacing frozen storage (i.e. tape), stuff that you rarely need to get access to, but higher tiers it'd be tough. It seems the trend is storing more and more data online, analytics and stuff.

Spoken as someone who has used S3 and other AMazon cloud things (ugh) for the past 1.5 years.

I think HP and Dell and stuff can do well in cloud storage, they know what they are doing, one thing(among countless many) Amazon does poorly is they do not offer tiers of storage, you can't pay them more to put more important data on enterprise storage (e.g. for database). They are introducing(or have recently introduced) SSD storage but I'm sure it's just a bunch of SSDs in a crapbox server like the rest of their infrastructure. This tiering doesn't really apply to S3 though, only to their other offerings.

My only concern for the likes of HP/Dell/etc building their own clouds is they may end up alienating their larger customers as they go head to head competing with them in the same market. Sort of like Microsoft and the Zune..

mixed feelings

PXG makes a good point and I agree though the flip side is how much they may burn their customers in the process(those building clouds based on HP gear). Double edged sword. Sort of reminds me when MS went out and screwed their music partners with the Zune (I think? with the new music formats). Not that the MS music market was a very big one at the time, same goes for the cloud market today, not a lot of players in the space since it is very capital intensive to get the right stuff up and going.

did suse ever support ext4 on the server?

doing a quick search reveals they supported ext4 on SUSE 11 desktop, though don't see mention of server, did they just skip it ?

interesting that the latest one offers ext4 in read only mode only.

late 2012

Didn't RIM already say Blackberry 10 for phones would be coming late 2012?

http://features.techworld.com/mobile-wireless/3336591/five-big-blackberry-announcements-look-forward/

"Though BlackBerry 7.1 certainly isn't BlackBerry 10, RIM's upcoming OS that's built on a new and completely different software foundation, it's definitely worth the update, and it ought to help tide over anxious CrackBerry users until BlackBerry 10 becomes available in late 2012 or early 2013."

Which to me means probably late Q1 2013. I recall last year many people were commenting how the new mobile OS would be out in Q1 2012.

Sad to see RIM crash and burn so much, their current decline combined with a disruptive upgrade I think will only cause further erosion in their market share in the next few years. Really doesn't seem that long ago that RIM's stock was a market darling reaching to new heights every few days/week.

http://business.financialpost.com/2012/02/07/halliburton-abandons-blackberry-picks-up-iphone/

"Halliburton Co. became the latest major enterprise customer to abandon RIM on Tuesday in favour of Apple’s iPhone, the Canadian company’s largest rival. The Houston, Texas-based firm — among the largest energy services providers in the world – plans to replace about 4,500 company-issued BlackBerrys with iPhones within two years."

not that I am a fan of apple, quite the opposite in fact. My smart phone platform of choice is WebOS so I know how RIM feels.

STP obsolete for over a decade

STP has been obsolete for over a decade, multiple protocols from multiple vendors are have been good alternatives to STP long before TRILL was a twinkle in anyone's eye. It's kind of funny and perhaps sad that the majority of folks don't seem to have realized this.

also the article doesn't mention the use of 802.3ad as a stopgap for faster ethernet, You can combine at least 8 links together to form a logical connection, with MLAG I think you can at least double that number if I recall right. This is of course widely deployed now and has been for years/decades(except MLAG which is kinda new)

"It's possible, we suppose, but unlikely that new Ethernet standards will evolve to cope with this, ether by specifying latency at a switch level or at a network level. "

We have that today too, it's called routing protocols, what do you think this hyper scale network with tens of thousands of switches would be a big flat layer 2 network? what a disaster that would be!

Be mindful of the costs of 100Gbit ethernet too. If your running a cross pacific undersea cable then costs for the optics probably aren't a big deal, but for most other sorts of deployments (intra data center at least) 100Gbit ethernet just isn't worth it today from a cost perspective.

netflix doesn't have porn

if it did I'd still be a subscriber

hot swap, online upgrade ?

are those redundant components hot swappable ? are there online software upgrades on the box or are those things you need the 'application high availability' for -- if so then there's no new street cred here.

HP overtook NetApp last year

according to the last report anyways

http://www.theregister.co.uk/2011/03/07/hp_passes_netapp_in_is_rankings/

"IDC's numbers for the fourth quarter (Q4) of 2010 list EMC in top position with a 26 per cent share of the revenue at $1,582m, IBM is second with a 16.3 per cent share ($996m), and HP third with $704m revenue and an 11.6 per cent share. NetApp, which was in this position in the third quarter, has dropped to fourth position with $630m revenue and a 10.3 per cent share."

another reg article and no mention of X-series

Another article about networking and no mention of the Black Diamond X-Series huh. 768 line rate 10GbE or 192 line rate 40GbE in 1/3rd of a rack. Twice the switching fabric vs the next closest competitor solution, half the power. The new Dell Force10 Leaf/Spine system draws a full 4 times more power and easily 3 times more rack space for the same sort of setup). Next generation fabric interconnect. Very cost effective. The X-series powered the SC11 conference not long ago.

Ethernet has powered the hyper scale cloud since the inception of hyper scale clouds, so the statement that ethernet can't do it doesn't make any sense. More than 40% of the top 500 super computers are powered by gigabit ethernet.

You've been able to build networks without STP for many years. I've been doing it for 8 years myself. Protocols have been around for ages, there are a lot of protocols whether things like ESRP, VSRP (not to be confused with VRRP), EAPS, M-LAG are the ones I can name off the top of my head at least, and I don't consider myself a network engineer by trade.

would of been nice

To see them go with at least a 3 controller design for N+1 redundancy(or at least give the customer the option given the cost of the controller is likely to pale in comparison to cost of the flash). Wonder how much of a performance hit there is if one of the controllers goes down.

you get what you pay for

I'm not in the UK so have never really heard of fasthosts but looked them up and looks like one of those cheap ass hosting providers.

Those who use them and get mad when there is downtime have nobody to blame but themselves.

Here I am paying US $100/mo to co-lo my 1U server for *personal* email/web hosting, I consider that cheap too.

stupid customers, fasthosts is for kids!

which is why

which is why I continue to shamelessly beg vmware to unlock vSphere standard edition, give us at least the same amount of vRAM tax as vSphere enterprise plus, if not unlock it entirely. I'll get along just fine with just HA + vMotion, the rest of the stuff is not important to me but I'm forced into those higher tiers of licensing due to arbitrary decisions on hardware scalability in the licensing.

waiting for KVM to mature more....hoping that can be a solution. Though it will be hard to switch away from vmware after having used them for 12 years.

almost got it

While the raw perf numbers are good it would of been nice to see Avere use a "real" storage system as their mass storage. I can't imagine customers deploying what must be at least $1M of Avere gear in front of some Opensolaris boxes which are probably not HA.

based on this article

http://www.theregister.co.uk/2011/09/19/avere_threatens_netapp/

And given that this seems to be in response to a NetApp number, it would of been neat to see Avere use a pair of FAS2000 arrays or perhaps a single FAS3200 array as the back end storage.

only wish SpecSFS forced disclosure of costs of the systems under test.

direct attached

I don't think direct attached virtualization requires high end hardware for HA.

What you need is fault tolerant software. Something that is scale out rather than scale up is ideal for direct attached virtualization. Where if a physical server with 20 VMs die you don't care.

A couple companies ago a decent portion of our web servers were deployed this way. Some of the core web apps could not scale beyond 1 physical CPU. Rather than try to re-write the code it was simpler to put a hypervisor on a dual proc quad core box and run 8 copies of the app (with 4 servers/site or roughly 32 copies of the app per active-active site[multiple sites for reduced customer latency]). Another app was higher performance and could leverage the underlying hardware to it's full extent so that web application ran on bare metal. The cost of a good shared storage array was going to outweigh the cost of all of the rest of the equipment at each site, so it didn't make a lot of sense or cents from a cost stand point, as much as I would of jumped at the opportunity to have such a system for an ease of use standpoint.

You forgot to mention the hybrid approach., ala vSphere 5 VSA or things like HP Lefthand VSA which turn direct attached storage into fault tolerant network storage - good for SMBs with low I/O needs, can get the best of both worlds.

I don't think your storage numbers are realistic either, measuring storage based on throughput when virtualization is for a large part random I/O workloads rendering throughput numbers meaningless, it's all about IOPS, the network is not the bottleneck. Sure there are some workloads that are throughput based but in the vast majority of cases your going to run out of IOPS long, long before you even get close to running out of bandwidth(consider, peaking out at maybe 4-8MB/second/disk at 15k RPM if your lucky). Latency is just as important as IOPS too. If your truly throughput bound then you may be better off running on bare metal. Hypervisors don't make sense for everything.

Depending on the organization you can design your network up front so that once your hypervisors are deployed and your virtual switches are configured you rarely have to touch them again. For my last VM deployments I can count the number of times I had to configure a vSwitch after I installed the hypervisors on one hand (~60 VM hosts, several different clusters). My new VM infrastructure which is going in early next month is planned so I don't expect to have to touch the vSwitches for the lifetime of the product - at least 2-3 years. Not that it's a big deal if I have to, but if I don't need to then so much the better. Anything can happen but my experience tells me vSwitch configuration changes are few and far between.

Memory (capacity not performance) is the driving force behind virtualizaiton, which is why Vmware added the vTAX in v5. Memory availability is just as important in these big boxes making technologies such as HP Advanced ECC and IBM Chipkill absolutely critical to any VM deployment. ECC by itself is not enough, and has not been for years.

If mainframes were so good then why is IBM using KVM + Red Hat for their developer cloud instead of running Linux on mainframes (IBM used to advertise a lot about leveraging the multi tennant abilities of mainframes they don't seem to advertise that nearly as much anymore I haven't seen such an ad in years). IBM after all unlike anyone else has got to have the lowest cost of operating their very own gear, and I'm sure licensing their own software comes at no cost as well. I wrote about this a while back, the IBM developer cloud was focused around Java apps so from a technical perspective it wouldn't matter what platform they ran it on.

What do people use when milliseconds can cost millions of dollars? More often than not these days it seems to be overclocked x86-64 systems(El Reg has many articles on this). Mainframes are what you use when you can't tolerate downtime.