There seems to be an issue in FsCsMvc41WebApp.fsproj, as it has a GAC reference to MVC:
<Reference Include="System.Web.Mvc, Version=18.104.22.168, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL" />
Instead, it needs to have a NuGet reference, as in FsCsMvc41Web.csproj:
<Reference Include="System.Web.Mvc, Version=22.214.171.124, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
The causes the build to break when MVC4 is not in the GAC. See this thread for reference: https://github.com/projectkudu/kudu/issues/137
Hmmm, along the same line but not quite the same: the fsproj has a reference to ..\..\packages\System.Net.Http.2.0.20710.0, but it doesn't look like that NuGet package actually gets installed. Same for the other System.Net.Http.* packages.
Also, interestingly, the F# project doesn't have a packages.config at all, but seems to rely on the packages that came with the C# proj.
It should show up under Visual F# | ASPNET with the name "F# and C# Web Application (ASP.NET MVC 4)". Check the installed extensions (Tools | Extension Manager...) and verify that this extension is enabled.