EventHubTrigger - Functions 2.0 does not start - Could not load FunctionsProj.dll

[manums]

Investigative information

I was able to run function app which has eventhub binding from local as well as from West US2 Consumption plan. From a day, after we moved all our dependent projects to build in x64 and updating all nugets to latest - function app starts failing. It does not even start, it fails to load the functionsproject.dll itself.

I'm kind of stuck without being able to debug the issue. * This happens once in a while locally but had not impacted deployed azure function (on .NET core 2.1) *

• I'm looking for guidance on troubleshooting and fixing these issues once for all. *

Exception Message:

Could not load file or assembly 'Microsoft.DynamicsFraudProtection.DataFlow.EventHubConsumers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.

Exception StackTrace:

   at System.Runtime.Loader.AssemblyLoadContext.LoadFromPath(IntPtr ptrNativeAssemblyLoadContext, String ilPath, String niPath, ObjectHandleOnStack retAssembly)
   at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
   at Microsoft.Azure.WebJobs.Script.Description.FunctionAssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath, Boolean addProbingPath) in C:\azure-webjobs-sdk-script\src\WebJobs.Script\Description\DotNet\FunctionAssemblyLoadContext.cs:line 260
   at Microsoft.Azure.WebJobs.Script.DependencyInjection.ScriptStartupTypeLocator.<>c__DisplayClass8_1.<GetExtensionsStartupTypesAsync>b__0(AssemblyName assemblyName) in C:\azure-webjobs-sdk-script\src\WebJobs.Script\DependencyInjection\ScriptStartupTypeLocator.cs:line 120
   at System.TypeNameParser.ResolveAssembly(String asmName, Func`2 assemblyResolver, Boolean throwOnError, StackCrawlMark& stackMark)
   at System.TypeNameParser.ConstructType(Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.TypeNameParser.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.Type.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase)
   at Microsoft.Azure.WebJobs.Script.DependencyInjection.ScriptStartupTypeLocator.<GetExtensionsStartupTypesAsync>d__8.MoveNext() in C:\azure-webjobs-sdk-script\src\WebJobs.Script\DependencyInjection\ScriptStartupTypeLocator.cs:line 97
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Azure.WebJobs.Script.DependencyInjection.ScriptStartupTypeLocator.GetStartupTypes() in C:\azure-webjobs-sdk-script\src\WebJobs.Script\DependencyInjection\ScriptStartupTypeLocator.cs:line 58

Please provide the following:

• Timestamp:
• Function App version (1.0 or 2.0): 2.0
• Function App name: Local
• Function name(s) (as appropriate):
• Invocation ID:
• Region:

Repro steps

Provide the steps required to reproduce the problem:
a. Unable to start the app locally. after enabling all CLR exceptions I see an exception while loading EventHubConsumers.dll (functions project) itself which contains the azure functions. It is the same issue when I try to publish to consumption plan too.
It was working till we updated projects to use latest nugets.

• Tried updating to .NET cor 2.2 (did not help)
• Switched between VS 2017 and VS2019 (that did not resolve issue either)
• Downgraded Eventhub extensions from 3.0.6 to 3.0.5
• tried downgrading from Net.SDK functions to 1.0.28 from 1.0.29

Nugets:

Expected behavior

Provide a description of the expected behavior.

Actual behavior

Provide a description of the actual behavior observed.

Known workarounds

Provide a description of any known workarounds.

Related information

Provide any related information
C#
.Net Core 2.1
x64 build

Bindings used:
EventsHubTrigger

Packages used in project:

<PackageReference Include="Microsoft.ApplicationInsights" Version="2.10.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.0.0" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.EventHubs" Version="3.0.5" />
    <PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="2.2.0" />
    <PackageReference Include="Microsoft.Jarvis" Version="20190426.1.0-rc" />
    <PackageReference Include="Microsoft.NET.Sdk.Functions" Version="1.0.28" />
    <PackageReference Include="Newtonsoft.Json" Version="12.0.2" />
    <PackageReference Include="Polly" Version="7.1.0" />

Exception Details:

Azure Functions Core Tools (2.7.1434 Commit hash: eac7c28)
Function Runtime Version: 2.0.12555.0
.NET Setting AzureWebJobsStorage = UseDevelopmentStorage=true
.NET Setting ConsumerGroup = knowledgebuilder
.NET Setting WEBSITE_HOSTNAME = localhost:7071
.NET Setting AzureWebJobsScriptRoot = D:\Path\EventHubConsumers\bin\x64\Debug\netcoreapp2.1
.NET Setting AZURE_FUNCTIONS_ENVIRONMENT = Development
[8/1/2019 6:57:37 AM] Building host: startup suppressed:False, configuration suppressed: False

[manums]

• This is happening on both VS 2019 and VS2017.
• AzureFunctionTools are at latest 2.28 version.
• Functions project is build and run with x64 configuration on .NET core 2.1.
• All dependencies are also built with x64 configuration.

I'm still unable to make it work locally. Can someone take a look at this?

[manums]

More details from Functions Console:

Azure Functions Core Tools (2.7.1505 Commit hash: eb81829)
Function Runtime Version: 2.0.12590.0
[8/4/2019 10:09:25 PM] Building host: startup suppressed:False, configuration suppressed: False
[8/4/2019 10:09:26 PM] A host error has occurred
[8/4/2019 10:09:26 PM] System.Private.CoreLib: Could not load file or assembly 'Microsoft..EventHubConsumers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.
Value cannot be null.
Parameter name: provider
Press any to continue....[8/4/2019 10:09:27 PM] Building host: startup suppressed:False, configuration suppressed: False
[8/4/2019 10:09:27 PM] A host error has occurred
[8/4/2019 10:09:27 PM] System.Private.CoreLib: Could not load file or assembly 'Microsoft..EventHubConsumers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.
Hosting environment: Production
Content root path: D:**\EventHubConsumers\bin\x64\Debug\netcoreapp2.1
Now listening on: http://0.0.0.0:7071
Application started. Press Ctrl+C to shut down.
[8/4/2019 10:09:29 PM] Building host: startup suppressed:False, configuration suppressed: False
[8/4/2019 10:09:29 PM] A host error has occurred
[8/4/2019 10:09:29 PM] System.Private.CoreLib: Could not load file or assembly 'Microsoft.**.EventHubConsumers, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.
[8/4/2019 10:09:33 PM] Building host: startup suppressed:False, configuration suppressed: False

[brettsam]

Followed up on this internally -- but just to make sure this is public:

We can’t load one of your dependencies…

• In the GitHub issue you say you switched to x64 and it failed… what happens when you switch back to x86?
• What is Microsoft.DynamicsFraudProtection.DataFlow.EventHubConsumers and where does it come from?
• Is that dependency in your bin folder?
• Are you able to use that dependency in a non-Functions x64 project?
• Does anything change if you move up to Microsoft.NET.Sdk.Functions 1.0.29?

[ghost]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

[manums]

I could not follow up on this thread earlier. I could get these issues resolved. I have explained things which I followed on stackoverflow page - here.