When should a script (or more likely, function) output raw data, and when should it “massage” its output? The classic example is something like disk space. You’re querying WMI, and it’s giving you disk space in bytes. Nobody cares about bytes. Should your function output bytes anyway, or output megabytes or gigabytes? If you output raw data, how would you expect a user to get a more-useful version? Would you expect someone running your command to use Select-Object on their own to do the math, or would you perhaps provide a default formatting view (a la what Get-Process does) that manages the math? The “Microsoft Way” is to use a default view - again, it’s what Get-Process does. But views are separate files, and they’re only really practical (many say) when they’re part of a module that can auto-load them. What do you think? [boilerplate greatdebate]
Had an opportunity to speak with some folks on the Azure team yesterday - Mark Russinovich was kind enough to make a contact for me. First of all, fun fact: Azure only charges you for used pages in VHDs. That is, if you create a 100GB VHD and load 1GB of data on it, you’re paying for 1GB of data. Very clever. So it’s charging you as if it was a dynamically expanding VHD, but of course it’s a fixed VHD with all of the related performance improvements. Nice. Second, they basically confirmed something I’d suspected. Azure’s “website model” tends to appeal more to smaller businesses or personal Web sites; most “serious” players (my word) are using the IaaS model, meaning they’re hosting VMs in the cloud, not just hosting a Web site. Having a full VM under your control obviously has advantages in terms of management, along with the ability to run things like in-memory caching software, load additional Web extensions, and so on. IaaS is absolutely the right model for PowerShell.org for many of those reasons. That said, they also confirmed that the Web site model and the IaaS model cost about the same, at least as you get started. So it’s really - for a smaller Web site - a matter of what you want to do. Again, there are specifics about the IaaS model that work well for us, so that’s what we’re looking to do. Azure also costs about the same, in an apples-to-apples comparison, as Amazon Web Services. That’s probably somewhat deliberate on Microsoft’s part, but Azure has advantages. For one, their virtualization layer has been approved by the various Microsoft product teams, so if you’re running SharePoint or SQL Server in an Azure VM, the team will support you. Not the case with AWS. Also, I frankly found Azure’s presentation of the costs easier to grok. Fifth (I love numbered lists, sorry), I confirmed that the IaaS option charges you for (a) the VM’s you’re running, by the minute; (b) the storage used by all VM VHDs’ used pages, and (c) outbound bandwidth. This can potentially make IaaS more expensive than the “website” model because Azure won’t spin down an IaaS VM, so you run 24x7 unless you’re manually deallocating. With a website, Azure only spins up worker processes when they’re needed, so your site isn’t “running” 24x7, so you might pay less if it’s not being “hit” 24x7. Again, though, the website model offers us less control and flexibility. Just thought you’d enjoy some of those details!
So, I’ve gotten a two-VM version of PowerShell.org running in Azure. Yay, me! My *nix skills are unaccountably rusty (go fig), but it didn’t take too long. Restoring the WordPress installation was the toughest, as a number of settings had to be tweaked since the site is no longer under the same URL (the  test site that is).
Baseline
I ran a load test against the existing production site yesterday; you can view the results at http://loadimpact.com/load-test/powershell.org-1aa960e59c5972af12898b745de9ec49. This simulated a 50-person concurrent load from three US locations and on UK location, which approximates our real-world traffic. The results are what they are; we’re looking for the delta between these and the Azure-based system. In this test, the green line is the number of concurrent connections, and the blue is the time it took each page to load. The test ran for 10 minutes total, with each simulated user hitting three different pages on the site (home page, a forums topic, and a blog post). A key fact is that the site currently runs under a shared hosting plan; I don’t have any details on how much RAM, how much CPU, or what kind of bandwidth exists for the site. It’s also important to note that the production Web site uses a Content Delivery Network, or CDN, which offloads a good amount of traffic from the site proper. Because that costs, we didn’t implement a CDN for the test site. I’d therefore expect it to be somewhat slower.
But I bet they’re okay with Remote Desktop Protocol, right? And all those Remote Procedure Calls? And I bet they never even thought about why every *nix _system, ever, _has SSH enabled by default? But practically nothing else (by default)? Hmm.
This was a fascinating thing to see throughout The Scripting Games this year: _When exactly should you use Write-Verbose, and why? _The same question applies to Write-Debug.
“I use Write-Debug to provide developer-level comments in my scripts, since I can turn it on with -Debug to see variable contents.”
“I use Write-Verbose to provide developer-level comments in my scripts, since I can turn it on with -Debug to see variable contents.”
See what I mean? Some folks will suggest that Verbose is for “user-friendly status messages;” others eschew Debug entirely and prefer PSBreakpoints for that functionality. What guidance would you offer for using Write-Verbose and Write-Debug in a script? [boilerplate greatdebate]
I had no idea Azure gives MSDN subscribers a huge free monthly credit - $200 for the first month, and then on the Ultimate subscription level (which is what I get as an MVP) you get  $175 per month thereafter. That starts to really justify the MSDN pricing. You want a lab in the cloud? Free Azure! Given the free-ness of it, I decided to set up a PowerShell.org in the sky to see how it went. Configuring dual CentOS VMs was a bit of an all-day affair; I have less experience with RHEL (which is what CentOS is based on) and it took me a while to figure out that the built-in firewall was causing all my grief. Fixed now. Microsoft publishes some pretty good guides for getting a LAMP stack running on CentOS in Azure. Not great guides, but good. They lack a decent guide on getting Passive FTP working - and it’s a PITA because Azure only lets you configure incoming ports on a one-at-a-time basis (not ranges), and you can only have 25. So that’s kind of a pain. But I got it working, got MySQL installed and working, and I’m presently waiting on VaultPress to smush up our latest site backup and spew it onto the Azure server. Remember: you don’t pay for bandwidth going into Azure, so I can load the backup in as many times as I want without incurring bandwidth. This VaultPress thing is neat, if it works. It continually pulls changes from our WordPress installation and backs them up, timestamped, a la Apple Time Machine. Allegedly, if you give them the FTP info on you new server, and you have a base WordPress install working on the new server, they can “push” your whole site down to the new server. Given my fits and starts with FTP on CentOS today, we’ll see how well it works, but I’m optimistic. Dunno. It’s been saying “Testing Connection” for a long time now. Sigh. Anyway, I’m starting both VMs in extra-small instances. Part of what I want to play with is whether or not I can upgrade those to bigger instances without breaking the universe. Depends on how CentOS behaves when it suddenly finds itself running on “new hardware.” We shall see! If it works, then it’ll truly be killer in terms of scaling. I also want to see if we get more “juice” running two load-balanced extra-small instances vs. a small instance (which is technically twice as big as an extra-small). Common logic suggests that more, smaller servers is better - a la every web farm, ever. But it’ll be fun to test. **Question:** anyone have any Web site load-testing software they’re fond of? Mac or Windows is fine, or even both. I’ll enlist some folks to help with that, since I know my DSL line’s upstream side will chokepoint long before the Azure server does. Ooo, maybe we can have a PowerShell.org botnet that I could control… bwaa haa haa! Meantime, Eric Courville, our new volunteer Webmaster, is setting up a similar Azure-based VM set with his own MSDN subscription. In addition to documenting the setup process, we’re going to try and do some load-testing and see what kind of instances we need to run in to get solid performance out of the site. PowerShell.org currently peaks at fewer than 50-60 concurrent connections (and even that day was a rare peak), so we’ll load test to that number. Stay tuned!
When we started PowerShell.org, my company (Concentrated Tech) donated shared hosting space to get the site up and running. We knew it wouldn’t be a permanent solution, but it let us start out for free. We’re coming to the point where a move to dedicated hosting will be desirable, and we’re looking at the options. Azure and Amazon Web Services are priced roughly the same for what we need, so as a Microsoft-centric community Azure’s obviously the way to go. Azure Technical Fellow Mark Russinovich is having someone on his team connect with me to discuss some of the models in which we could use Azure. What makes the discussion interesting is that PowerShell.org runs on a LAMP (Linux, Apache, MySQL, and PHP) stack. We’re not looking to change that; WordPress requires PHP, and the Windows builds of PHP typically lack some of the key PHP extensions we use. I’m not interested in compiling my own PHP build, either - I want off-the-shelf. WordPress more or less requires MySQL; while there’s a SQL Server adapter available, it can’t handle plugins that don’t use WordPress’ database abstraction layer, and I just don’t want to take the chance of needing such a plugin at some point and not being able to use it. What’s neat about Azure is that it doesn’t care. I adore Microsoft for selling a service and not caring what I do with it. Azure runs Linux _just fine. _Huzzah! So, we’ve got two basic models that could work for us. Model 1 is to just buy virtual machines in Azure. We’re planning one for the database and another for the Web site itself, so that we can scale-out the Web end if we want to in the future. We’re not going to do an availability set; that means we risk some short downtime if Azure experiences hardware problems and needs to move our VM, but we’re fine with that because right now we can’t afford better availability. We’d probably build CentOS machines using Azure’s provided base image (again, adore Microsoft for making this easy for Linux hosting and not just Windows). We know we tend to top out at 250GB of bandwidth a month, and that we need about 1GB of disk space for the Web site. 500MB of space for the database will last us a long time, but we’d probably get 1GB for that, too. It’s only like $3 a month. We could probably start with Small VM instances and upgrade later if needed. All-in, we’re probably looking at about $125/mo, less any prepay discounts. Model 2 is to just run a _Website. _We still get to pick the kind of instance that hosts our site, so if we went with Small and a single instance, we’d be at about $110 including bandwidth and storage. That doesn’t include MySQL, though. Interestingly, Microsoft doesn’t host MySQL themselves as they do with SQL Azure. Instead, they outsource to ClearDB.com, which provides an Azure-like service for hosted MySQL. Unfortunately, the Azure price calculator doesn’t cover the resold ClearDB service. Looking at ClearDB’s own pricing, it’d probably push us to about $120-$125 a month - or about the same as having our own virtual machines. The difference is that, with Model 2, Microsoft can float our Web site to whatever virtual hosts they need to at the time to balance performance; with Model 1, they can potentially move our entire VM - although they’re unlikely to do so routinely, since it’d involve taking us offline for a brief period. A super-neat part of this model is its integration with Git: I can run a local test version of the site, and as I make changes and commit them to our GitHub repository, Azure can execute a pull and get the latest version of the site code right from Git. Awesome and automated. I love automated. An appeal of Model 1 is that I can build out the proposed CentOS environment on my own Hyper-V server, hit it with some test traffic loads, and size the machine appropriately. I can then deploy the VHDs right to Azure, knowing that the instance size I picked will be suitable for the traffic we need to handle. It also give me an opportunity to validate the fact that a dedicated VM will be faster than our current shared hosting system, and to play around with the more advanced caching and optimization options available on a dedicated VM. I can get everything dialed in perfectly, and then deploy. Azure has other usage models, but these are the two applicable to us. I think it’s great that we get these options, and that the pricing is more or less the same regardless. And again, I think it’s pure genius that Azure’s in the business of making money for Microsoft, and that they’re happy to do so running whatever OS I want them to. I’ll continue this series of posts as we move through the process, just for the benefit of anyone who’s interested in seeing Azure-ification from start to finish. Let me know if you have any questions or feedback!
_PowerShell in Depth_ and _Learn Windows PowerShell 3 in a Month of Lunches_ are on half-price August 25th, 2013. Use code dotd0825au at www.manning.com/jones2/ or Use code dotd0825au at www.manning.com/jones3/ Tell a friend who needs to start learning PowerShell - two great books at 50% off. All print books come with a voucher for free ebook versions (MOBI, EPUB, PDF), and the ebook-only version is also 50% off.
This weekend, we’ll be conducting maintenance on PowerShell.org. We have several goals: **New visual theme. **We’ll be installing a new visual theme. While we hope to catch everything, you may run across something goofy-looking. Please use the Community Discussion forum to report that, so we can ask the designers to take a look. **Performance. We’re going to continue to work on performance, with a goal of getting specified pages to have an “A” on the Page Test and YSlow tests. That’s not the entirety of performance, but it’s what we can address now without moving to a different hosting environment (which is planned). During this phase of our maintenance, the site may not function correctly, or certain features may come and go as we test different configurations. Cleanup. **We’ll be condensing certain features of the site, rearranging menus, and so on, to provide a better visual experience across a wider variety of devices. We appreciate your patience!
Spoke too soon in the morning’s updates; my designer buddies worked last night and took their first stab at the forums pages. They also changed their mind about the big black boxes, which I appreciate ;). The forums material is denser now, meaning more info per page, which should please some folks. Samples below - and comments welcome. Just keep in mind these folks aren’t being paid, so be nice ;).
