Future of Web Apps (FOWA)
Session: Languages don't scale
Note by Phil Wilson:
Also Joe Stump, chief engineer at Digg.
The problem with zealots. People actually put their hands up for who thinks their language is the best!
Talking about why their favourite languages suck. PHP, Ruby, Python.
Python has a dictator which is a problem. "You can't design my consensus, developers are little children"
Basically a slanging match on all the scripting languages :)
IO is important, and this is where the bottleneck is, so how your codes manages this is what's important (to a lesser extens how your language manages this). Speed of packets on the network is constant, you're not going to improve on the speed of light.
Enjoying the talk because it's obvious that the know what they're talking about and are real developers. Knowledge across multiple languages is great.
scaling out (RAI Computers, just like google always say) rather than scaling up. When blaine was at twitter, one machine loads of CPUs, 32GB RAM, digg has max 16GB ram in their machines. Scaling out means you need to ensure that all machines get equal load, no bottlenecks anywhere from front to back - database, web server. you should be able to turn off any machine and have everything work. if you can't, you're doing it wrong. Also, please don't use NFS.
Happy coders are productive coders. Making your coders happy is the best way to productivity.
Use whatever language you like, because making your coder happy is the most important thing, not the speed of the language. The ones that use CPU - who cares? it's 3% of Ruby usage in their demo, so a fast langauge reduces CPU usage and speeds up page loads my milliiseconds, which is irrelevant. IO is where it's at.
Cal Henderson hates test code. Probably because he's actually a good coder ;)
digg 1/2-1TB memcached, flickr 3TB, facebook 7TB cache.
if you're not using memcached, your app is slow. although for some operations, mysql is only fractionally slower. memcachedb is worth examining. (with 0 load it was doing 15,000 reads a second)
the problem is invalidating the cache. if it invalidates and all webservers (say 50) notice at the same time, that is a problem because they all make real queries. jittering is a solution that they use at digg.
everything should be using queuing (obv. since Blaine wrote starling). Digg use gearman. 3-5million jobs a day. anything that stops the user working is bad.
partitioning data - partitioning reads is easy, say half a day's work. write is much harder.
Pretty much everything will scale.
question about ops vs. developers. best devs have been ops. should always work closely with ops.
flickr once went down because it ran of disk space.
The problem with zealots. People actually put their hands up for who thinks their language is the best!
Talking about why their favourite languages suck. PHP, Ruby, Python.
Python has a dictator which is a problem. "You can't design my consensus, developers are little children"
Basically a slanging match on all the scripting languages :)
IO is important, and this is where the bottleneck is, so how your codes manages this is what's important (to a lesser extens how your language manages this). Speed of packets on the network is constant, you're not going to improve on the speed of light.
Enjoying the talk because it's obvious that the know what they're talking about and are real developers. Knowledge across multiple languages is great.
scaling out (RAI Computers, just like google always say) rather than scaling up. When blaine was at twitter, one machine loads of CPUs, 32GB RAM, digg has max 16GB ram in their machines. Scaling out means you need to ensure that all machines get equal load, no bottlenecks anywhere from front to back - database, web server. you should be able to turn off any machine and have everything work. if you can't, you're doing it wrong. Also, please don't use NFS.
Happy coders are productive coders. Making your coders happy is the best way to productivity.
Use whatever language you like, because making your coder happy is the most important thing, not the speed of the language. The ones that use CPU - who cares? it's 3% of Ruby usage in their demo, so a fast langauge reduces CPU usage and speeds up page loads my milliiseconds, which is irrelevant. IO is where it's at.
Cal Henderson hates test code. Probably because he's actually a good coder ;)
digg 1/2-1TB memcached, flickr 3TB, facebook 7TB cache.
if you're not using memcached, your app is slow. although for some operations, mysql is only fractionally slower. memcachedb is worth examining. (with 0 load it was doing 15,000 reads a second)
the problem is invalidating the cache. if it invalidates and all webservers (say 50) notice at the same time, that is a problem because they all make real queries. jittering is a solution that they use at digg.
everything should be using queuing (obv. since Blaine wrote starling). Digg use gearman. 3-5million jobs a day. anything that stops the user working is bad.
partitioning data - partitioning reads is easy, say half a day's work. write is much harder.
Pretty much everything will scale.
question about ops vs. developers. best devs have been ops. should always work closely with ops.
flickr once went down because it ran of disk space.
