Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concurrent BulkInsertOrUpdateAsync Calls with UpdateByProperties Set, throws Duplicate Key Name Errors #1634

Open
Seiori opened this issue Dec 10, 2024 · 4 comments
Labels

Comments

@Seiori
Copy link

Seiori commented Dec 10, 2024

I have an import script that runs 3 concurrent instances, each perform a BulkImportOrUpdateAsync call with the UpdateByProperties value set to my unique Key of my object.

However; when running this routine, I will occassionally get errors in regards to a Duplicate Key Name on the Temp Key it tries to create using the UpdateByProperties field. (This field does have a Unique Key already set)

Is there some property I can set so that the Temp Keys created to have a random identifier like the Temp Tables. Or could this be implemented if not?

await context.BulkInsertOrUpdateAsync(distinctSummonersList, options =>
{
    options.UpdateByProperties = [nameof(Summoners.Puuid)];
});
public class Summoners
{
    [Key] public uint Id { get; set; } 
    [MaxLength(78)] public string Puuid { get; set; }
    [MaxLength(64)] public string SummonerId { get; set; }
    public PlatformRoute Platform { get; set; }
    [MaxLength(43)] public string RiotId { get; set; }
    public ushort ProfileIcon { get; set; }
    public ushort SummonerLevel { get; set; }
    
    // RowVersion property
    [Timestamp]
    public byte[] RowVersion { get; set; }
    
    public ICollection<SummonerRanks> RankedEntries { get; set; }
    public ICollection<Participants> MatchHistory { get; set; }
}
modelBuilder.Entity<Summoners>()
    .HasKey(s => s.Id);

modelBuilder.Entity<Summoners>()
    .HasIndex(s => s.Puuid)
    .IsUnique();

modelBuilder.Entity<Summoners>()
    .HasIndex(s => new { s.Platform, s.SummonerId })
    .IsUnique();

modelBuilder.Entity<Summoners>()
    .Property(s => s.Platform)
    .HasConversion<short>();

modelBuilder.Entity<Summoners>()
    .Property(s => s.RowVersion)
    .IsRowVersion();
@borisdj
Copy link
Owner

borisdj commented Dec 10, 2024

Are you using SqlServer?, since there as far as I know Temp table does not have nor created any Key or Indexes.
If that's what you are reffering to.

@Seiori
Copy link
Author

Seiori commented Dec 10, 2024

I am using MySQL, I did transition over to SQLServer briefly and didn't experiance this issue which I think is because of what you mentioned here. But decided in the end to revert back

@Seiori
Copy link
Author

Seiori commented Dec 12, 2024

Thinking about it, is this intended? I already have a Unique Key set against my Puuid column. Is there a reason why it needs to setup a Temp Key against that column again for the same table.

I am wondering if my configuration is perhaps wrong?

I have defined it as a Unique Index within my model builder. Is there anything else I can do so that it uses this Key rather than trying to create a new one.

@Seiori
Copy link
Author

Seiori commented Dec 13, 2024

From my understanding the issues are likely one or more of the following:

  • My Unique Key on my Puuid Column is not being picked up. Resulting in the code attempting to create a temporary Unique Key against the table that already has a Unique Key. Possible configuration issue on my side in the Table, C# Object, or EF Core Model Builder. What are your suggestions to handle this?
  • The "GetUniqueConstrainName" function within "MySqlQueryBuilder" only creates a Key Name that is Unique Outside of Concurrency. In my case because I have 3 different instances interacting with my Summoners Table. All of them are generating the same "Unique" Constrain Name. If this was actually Unique with some randomness inserted this would likely solve the issue.

Curiously, if I want to run my code concurrently, what are the best practices you suggest?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants