How do you get a numeric-only string and ignore all non-numeric characters, using just standard SQL functions? There’s SQL replace and pattern functions, but they only work once on a string. So barring 53 nested REPLACE functions, it’s tough. I’ve run into this issue many times before while writing import scripts and the like.

Today, I had a stroke of brilliance – what if it could be done using FOR XML and spt_values to parse the entire string? A bit of googling and stack overflowing later, and some gluging of different parts together, and I came up with the following:

SELECT 
    (SELECT CAST(CAST((
        SELECT SUBSTRING(FieldToStrip, Number, 1)
        FROM master..spt_values
        WHERE Type='p' AND Number <= LEN(FieldToStrip) AND
            SUBSTRING(FieldToStrip, Number, 1) LIKE '[0-9]' FOR XML Path(''))
    AS xml) AS varchar(MAX)))
FROM
    SourceTable

Replace SourceTable with your table and FieldToStrip with your field, and away you go. If you want to include other allowed characters, change the pattern in the LIKE -- that specifies which characters to retain. All other fields will be removed.

Every time I need to set up a new server with Web Deploy/MSDeploy, I end up having to do a bunch of googling and tweaking and it never ends up being simple. The error messages are so arcane it's hard to know what the issue is when it doesn't work. There are several different ways to set up Web Deploy as well, from the IIS management service to a separate web deploy service. It’s fairly straightforward once you’ve got a good procedure to follow, but there are so many moving parts and the docs are horrible – fragmented blogs strewn everywhere each with a piece of the puzzle.

This most recent time I put together a comprehensive guide while I was working through the configuration.

First, set up IIS remote management services. You can do this through the UI:

  • Open role manager
    • Web Server -> Add Role Services
    • Management Tools -> Management Service
  • Open IIS Manager
    • <servername> -> Management Service
      • Check "Enable Remote Connections"
      • Check "Windows or IIS Manager"

Or batch file commands:

dism /online /enable-feature /featurename:IIS-WebServerRole
dism /online /enable-feature /featurename:IIS-WebServerManagementTools
dism /online /enable-feature /featurename:IIS-ManagementService

reg Add HKLM\Software\Microsoft\WebManagement\Server /V EnableRemoteManagement /T REG_DWORD /D 1

net start wmsvc
sc config wmsvc start= auto

Then, set up the Web Deploy service and connect it to IIS Manager:

  • Install web deploy from http://www.iis.net/downloads/microsoft/web-deploy
    • Make sure to install the deployment handler with delegation and non admin
  • Open IIS Manager
  • In the server configuration, add an IIS Manager user
  • Go to each website, IIS Manager Permissions and give user permissions
  • Add Web Management Service (wmsvc) permissions for each physical location where the site files will be located (Local Service, or whatever user wmsvc is running under)
  • Set WmSvc to start auto
  • (re)Start wmsvc
  • Open port 8172 on all relevant firewalls. If you're using Windows Firewall, you can use this:
    netsh firewall add portopening TPC 8172 WdeployAgent

Finally, set up a delegation rule to allow users to create applications. This is called a "Mark Folders as Applications" rule, using the createApp provider. If your applications are already existing and won't be created during deploy, you still need this because the rule is used required to verify that the application exists. You can skip the modify grant permission on applicationHost.config if you'll never be actually creating an application during deploy.

  • Create a user account (I call it "CreateAppUser")
  • Grant read permission to %windir%\system32\inetsrv\config.
  • Grant modify permission to %windir%\system32\inetsrv\config\applicationHost.config. Skip this step if you won't be creating any applications during deploy
  • Open Management Service Delegation in IIS Manager
  • Add a Mark Folders as Applications Rule
  • Set the CreateAppUser as the Run As user, using a Specific User type
  • Click OK and a Add User To Rule dialog will come up
  • Enter * in the Name and click OK. This will allow all users to create applications.

And… done!

The first time you open VS and try to publish, go through the whole wizard to make sure settings are correct. You'll likely get a popup about the untrusted certificate used by IIS Manager -- accept the certificate.

By default, SQL Server 0’s out database files as it allocates space for them. This can be good for security/reliability in a production environment, but it means SQL Server is writing a bunch of extra data. Turn on instant file initialization in dev/stage/test environments for dramatically faster database creation and backup/restore operations.

There’s no switch for this in SQL Server, strangely enough. To get instant file initialization enabled, you’ve got to grant the “Perform volume maintenance tasks” permission to the SQL Server service account.

Brad McGehee’s got a great detailed analysis of what happens with instant file initialization and walkthrough steps with screenshots.

Microsoft’s Dynamic Linq library has been floating around the internet in one form or another for years. Its been distributed as a raw .cs file, not a NuGet or any sort of supported package. The most central location I’ve found for this is King Wilder’s System.Dynamic.Linq GitHub repo and NuGet package.

I needed support for a couple of things that weren’t included in the original version, namely Sum/Average/Min/Max functions, and that in a dynamically callable way. So I built that, cribbing off of a Sum function found on StackOverflow. The code takes the name of the function you want to call and the name of the property you want to aggregate. It then uses reflection to grab the extension method of the correct type based on the property type you specify and executes it against the IQueryable, returning the result.

After a bit of jiggering, I realized that Sum and Average have totally different signatures than Min/Max for some reason. Adding in separate branches to the code did the trick and I came up with:

/// <summary>
/// Dynamically runs an aggregate function on the IQueryable.
/// </summary>
/// <param name="source">The IQueryable data source.</param>
/// <param name="function">The name of the function to run. Can be Sum, Average, Min, Max.</param>
/// <param name="member">The name of the property to aggregate over.</param>
/// <returns>The value of the aggregate function run over the specified property.</returns>
public static object Aggregate(this IQueryable source, string function, string member)
{
    if (source == null) throw new ArgumentNullException("source");
    if (member == null) throw new ArgumentNullException("member");

    // Properties
    PropertyInfo property = source.ElementType.GetProperty(member);
    ParameterExpression parameter = Expression.Parameter(source.ElementType, "s");
    Expression selector = Expression.Lambda(Expression.MakeMemberAccess(parameter, property), parameter);
    // We've tried to find an expression of the type Expression<Func<TSource, TAcc>>,
    // which is expressed as ( (TSource s) => s.Price );

    var methods = typeof(Queryable).GetMethods().Where(x => x.Name == function);

    // Method
    MethodInfo aggregateMethod = typeof(Queryable).GetMethods().SingleOrDefault(
        m => m.Name == function
            && m.ReturnType == property.PropertyType // should match the type of the property
            && m.IsGenericMethod);

    // Sum, Average
    if (aggregateMethod != null)
    {
        return source.Provider.Execute(
            Expression.Call(
                null,
                aggregateMethod.MakeGenericMethod(new[] { source.ElementType }),
                new[] { source.Expression, Expression.Quote(selector) }));
    }
    // Min, Max
    else
    {
        aggregateMethod = typeof(Queryable).GetMethods().SingleOrDefault(
            m => m.Name == function
                && m.GetGenericArguments().Length == 2
                && m.IsGenericMethod);

        return source.Provider.Execute(
            Expression.Call(
                null,
                aggregateMethod.MakeGenericMethod(new[] { source.ElementType, property.PropertyType }),
                new[] { source.Expression, Expression.Quote(selector) }));
    }
}

I've submitted a pull request with this function to kahnau's repo, and he seems pretty responsive about accepting pull requests, so this will likely be in that NuGet soon.

Software is a unique industry because it is constantly self-improving its own productivity. It acts as an accelerating concept like interchangeable parts or the assembly line, rather than a linear incrementing body of study. The use of software applies that acceleration to everywhere it is used, including itself.

So why don't we have lower skilled people building software by drag/dropping building blocks as many have predicted? After all, we have that self accelerating improvement applied to software over decades. Exponential increases in programmer productivity would suggest that lower skilled people could be employed, dropping average salaries, right? That kind of logic leads toward thinking there is a tech worker shortage.

There is no "tech worker shortage". Companies might not be able to find low priced labor, which they believe they should be able to do based on the reasoning above. But like the linked article quotes: "I'm not sure that qualifies as a shortage, any more than my not being able to find a half-priced TV."

What's wrong with this train of thought?

First off, there are many "building block" type software packages available. It's just hard to see them because the demand for new and more complex software has grown so dramatically. There are website builders, time tracking software, component based development tools galore. They've satisfied an enormous amount of what people wanted to do 20 years ago. But in the meantime, demand and complexity of requirements have grown exponentially. The acceleration of productivity is just barely allowing the software industry to run in place.

At the same time, there is an increasing demand for custom built software that is tailored directly to the data and business logic of the client, particularly in the LOB arena. It used to be too expensive for anything other than massive companies to afford custom development. Now, productivity increases allow even small businesses to have staff or contract programmers building software to enhance their business, again, pushing demand for programmers back up.

Finally, and most importantly, software is getting more complex in general. The requirements are bigger, in features, volume, and reliability. The software enabling productivity increases is sitting on layer upon layer of other software packages. 20 years ago, you'd build a simple desktop app in C++ or VB. Or a CGI app for web. It'd be a couple of components, sitting on a single server (or maybe a cluster of two machines if you were really rich and important), with a simple database. The "cloud" was still a twinkle in Jeff Bezos's eye until less than 10 years ago. Now, you have an MVC app using 20 different packages imported by a package manager running on a load balanced set of web servers, a database sharded across several servers, some memcache servers on the backend to accelerate data access, some varnish caches on the front to accelerate static pages, some CDN to accelerate content. Etcetera etcetera.

A senior software engineer that knows all of the possible tools and can orchestrate them together, someone that knows the concepts from the underlying cpu/cache/memory up to the networked cloud, and can use the right tool (out of 100s) for the right job, getting the job done (not to mention done right, and done fast) -- that person's value and productivity is multiplied by the layer upon layer of tooling developed over the past couple of decades. They can do the work that 10 senior people did 10 or 15 years ago, or 100 junior devs, while the junior dev's productivity has remained about the same.

That dichotomy between productivity of a senior and junior is what is throwing big companies for a loop. Their mental model of staffing is hierarchical, with a couple of seniors at the top, then some mids, and a bunch of juniors filling out the crowd. Their sort of thinking around cost is what leads to outsourcing to low skilled body shops. They can't fathom that it would be cheaper and more effective to hire a bunch of seniors and a couple mids to get the job done right -- they just see the dollar signs on the salary packages and scream "tech shortage".

Software development is becoming another market with winner-take-all dynamics. Sure, anybody can get started easy and for free. But the thousands of hours invested over years that it takes to gain an innate understanding of the innumerable tools and techniques involved? That’s another matter entirely.