Skip to content

Build-Time Analyzers

NPipeline ships with Roslyn analyzers that catch bugs at build time - before your pipeline ever runs. Misconfigured retry policies, blocking calls in async nodes, LINQ in hot paths, sinks that silently drop data - the analyzers flag all of these as compiler warnings or errors with automatic code fixes.

This is a key differentiator: most data pipeline libraries only fail at runtime. NPipeline catches entire categories of mistakes during dotnet build.

Installation

The analyzers are distributed as separate NuGet packages and must be added explicitly:

bash
dotnet add package NPipeline.Analyzers

Connector-specific analyzers are also separate packages:

bash
dotnet add package NPipeline.Connectors.Postgres.Analyzers
dotnet add package NPipeline.Connectors.SqlServer.Analyzers

NP90xx - Configuration and Setup

RuleSeverityTitleFix
NP9001WarningRestartNode decision requires complete resilience configurationReview resilience configuration when ResilienceDecision.RestartNode can be returned. Verify that MaxItemRetries, MaxNodeRestartAttempts, and MaxMaterializedItems are all properly configured.
NP9002ErrorUnbounded materialization configurationSet MaxMaterializedItems on PipelineRetryOptions to prevent out-of-memory crashes.
NP9003WarningInappropriate parallelism configurationReduce DegreeOfParallelism or disable PreserveOrdering when parallelism is high.
NP9004WarningBatching configuration mismatchVerify BatchSize and BatchTimeout are consistent with the node's expected throughput.
NP9005WarningInappropriate timeout configurationFix negative, zero, or excessively low timeout values.

NP91xx - Performance and Optimization

Rules NP9103–NP9107 are profile-gated: they only fire when the optimization profile is set to HighThroughput (via <NPipelineOptimizationProfile>HighThroughput</NPipelineOptimizationProfile> in your project file). In the Default profile, these rules are suppressed because their micro-optimizations are unnecessary at moderate throughput. Rules NP9101 and NP9102 are always active regardless of profile.

RuleSeverityTitleProfile-GatedFix
NP9101WarningBlocking calls in async methodsNoReplace .Result, .Wait(), GetAwaiter().GetResult(), Thread.Sleep() with async equivalents.
NP9102WarningSynchronous over async anti-patternsNoAvoid wrapping synchronous code in Task.Run() inside nodes. Use ValueTask fast paths instead.
NP9103WarningLINQ operation detected in hot pathYesReplace LINQ (Where, Select, ToList) in TransformAsync with foreach loops to avoid allocations.
NP9104WarningInefficient string operation detectedYesReplace string concatenation with + in loops with StringBuilder.
NP9105WarningAnonymous object allocation in hot pathYesReplace anonymous objects in TransformAsync with records or structs.
NP9106InfoConsider overriding ExecuteValueTaskAsync for synchronous operationsYesOverride ExecuteValueTaskAsync when TransformAsync uses Task.FromResult. See Synchronous Fast Paths.
NP9107WarningUse streaming patterns in SourceNode implementationsYesReturn new DataStream<T>(asyncEnumerable) instead of materializing everything with .ToList().
NP9108InfoAdd parameterless constructor for better performanceNoAdd a parameterless constructor for faster node activation.

NP92xx - Reliability and Error Handling

RuleSeverityTitleFix
NP9201WarningDo not swallow OperationCanceledExceptionDon't catch and suppress OperationCanceledException; let it propagate for proper cancellation handling.
NP9202WarningInefficient exception handlingAvoid catch (Exception) with rethrow in tight loops. Use specific exception types.
NP9203WarningMethod should respect cancellation tokenPass CancellationToken to async methods that accept it.

NP93xx - Data Integrity and Correctness

RuleSeverityTitleFix
NP9301ErrorSinkNode not consuming inputThe ConsumeAsync method must use its input parameter. A sink that ignores its input is always a bug.
NP9302WarningUnsafe PipelineContext accessAvoid concurrent writes to PipelineContext.Items from parallel nodes when using HighThroughput profile. In Default profile, dictionaries are thread-safe. Use IPipelineStateManager for complex shared state.

NP94xx - Design and Architecture

RuleSeverityTitleFix
NP9401InfoConsider IStreamTransformNodeFor transforms that operate on entire streams rather than individual items, implement IStreamTransformNode<TIn, TOut>.
NP9402WarningIStreamTransformNode should use IStreamExecutionStrategyIStreamTransformNode requires IStreamExecutionStrategy. Don't combine with per-item strategies.
NP9403WarningNode missing public parameterless constructorNode types resolved by the framework need a public parameterless constructor or DI registration.
NP9404WarningDependency injection anti-patternAvoid service locator patterns; use constructor injection instead.

NP95xx - Connector-Specific

RuleSeverityPackageTitleFix
NP9501WarningPostgresPostgreSQL source with checkpointing requires ORDER BY clausePostgreSQL sources using checkpointing must include an ORDER BY clause for deterministic replay.
NP9502WarningSQL ServerSQL Server source with checkpointing requires ORDER BY clauseSQL Server sources using checkpointing must include an ORDER BY clause for deterministic replay.

Automatic Code Fixes

Most analyzer rules include automatic code fix providers. Apply fixes individually via the lightbulb menu, or use Fix All to batch-apply across a project.

Key Code Fix Examples

NP9002 - Add MaxMaterializedItems:

csharp
// Before
new PipelineRetryOptions(MaxItemRetries: 3, MaxNodeRestartAttempts: 3)

// After (code fix applied)
new PipelineRetryOptions(MaxItemRetries: 3, MaxNodeRestartAttempts: 3, MaxMaterializedItems: 10000)

NP9101 - Replace blocking call with await:

csharp
// Before
var data = httpClient.GetAsync(url).Result;

// After
var data = await httpClient.GetAsync(url);

NP9107 - Use streaming source:

csharp
// Before
public override IDataStream<Record> OpenStream(PipelineContext ctx, CancellationToken ct)
{
    var records = db.GetAll().ToList(); // NP9107 fires here - materializing into List
    return new InMemoryDataStream<Record>(records, "records");
}

// After (manually refactored - wrap the async enumerable directly)
public override IDataStream<Record> OpenStream(PipelineContext ctx, CancellationToken ct)
    => new DataStream<Record>(db.GetAllAsync(ct), "records");

NP9301 - Consume sink input:

csharp
// Before
public override async Task ConsumeAsync(IDataStream<Order> input, PipelineContext ctx, CancellationToken ct)
{
    await db.SaveAsync(ct); // input never consumed - bug!
}

// After (manually implemented - always enumerate the input stream)
public override async Task ConsumeAsync(IDataStream<Order> input, PipelineContext ctx, CancellationToken ct)
{
    await foreach (var order in input.WithCancellation(ct))
    {
        await db.InsertAsync(order, ct);
    }
}

NP9404 - Replace service locator with constructor injection:

csharp
// Before
public class MyNode : TransformNode<In, Out>
{
    public override Task<Out> TransformAsync(In item, PipelineContext ctx, CancellationToken ct)
    {
        var service = ctx.Properties["ServiceProvider"] as IServiceProvider;
        var dep = service.GetRequiredService<IMyService>();
        ...
    }
}

// After (manually refactored - use constructor injection)
public class MyNode : TransformNode<In, Out>
{
    private readonly IMyService _dep;

    public MyNode(IMyService dep) => _dep = dep;

    public override Task<Out> TransformAsync(In item, PipelineContext ctx, CancellationToken ct)
    {
        ...
    }
}

Profile-Gated Analyzers

Five performance analyzers (NP9103–NP9107) are controlled by the <NPipelineOptimizationProfile> MSBuild property. When set to Default (the default), these rules are suppressed - their micro-optimizations are unnecessary at moderate throughput. When set to HighThroughput, all rules fire.

Set the property in your project file:

xml
<PropertyGroup>
    <NPipelineOptimizationProfile>HighThroughput</NPipelineOptimizationProfile>
</PropertyGroup>

This corresponds to the runtime PipelineBuilder.WithOptimizationProfile() setting. See Optimization Profiles for the full picture.

When profile metadata is available in your compiled assemblies (for example from NPipeline.Analyzers package props), NPipeline warns at runtime build if the MSBuild profile and runtime profile diverge.

Suppressing Rules

Suppress individual diagnostics when needed:

csharp
#pragma warning disable NP9103 // LINQ is acceptable here - called once, not per-item
var lookup = items.ToDictionary(x => x.Id);
#pragma warning restore NP9103

Or in .editorconfig:

ini
[*.cs]
dotnet_diagnostic.NP9106.severity = none  # disable ValueTask suggestion

Next Steps

Released under the MIT License.