您现在的位置是:网站首页> 编程资料编程资料

.Net极限生产力之分表分库全自动化Migrations Code-First_实用技巧_

2023-05-24 335人已围观

简介 .Net极限生产力之分表分库全自动化Migrations Code-First_实用技巧_

开始

本次我们的主题就是极限生产力,其他语言望尘莫及的分表分库全自动化Migrations Code-First 加 efcore 分表分库无感开发

,经过这么多框架的兼容我自己也认识到了一些问题,譬如在ShardingCore初始化前使用(毕竟efcore)的初始化是在依赖注入的时候不需要手动调用初始化,比如efcore.tool的迁移的问题,本项目不能迁移,因为efcore.tool在使用命令的时候不会调用Configure导致无法初始化的bug,导致迁移必须要通过新建控制台程序,而不能在本项目内迁移,再或者code-firstShardingCore的启动参数冲突导致需要平凡修改,并且不支持分库,之前有小伙伴分了300个库如果自动迁移不能用确实是一件很头疼的事情,虽然这些问题对于分库分表而言其实是小事情,但是如果一旦分表分库到达一定的量级就会难以维护。所以ShardingCore在最近三周内开启了新的版本,新版本主要是解决上述痛点并且将代码更加标准的使用

开发软件一般是先能用,然后好用,最后标准化,ShardingCore也是如此,因为需要扩展efcore所以有时候在不熟悉efcore的扩展方式的时候只能靠静态类来进行注入访问,而静态类其实是一个非常不标准的用法,除非万不得已。那么新版本x.6.x.x ShardingCore带来了什么请往下看

移除静态容器

静态容器的使用导致ShardingCore在整个应用程序声明周期只有一份数据,那么数据都是共享的这个对于后续的测试维护扩展是相当的不利的,没有单例那种隔离性来的好,所以移除了ShardingContainer,通过提供IShardingRuntimeContext来保证和之前的参数结构的访问,同一个DbContext类型在使用不同的IShardingRuntimeContext后可以表现出不同的分表分库特性。

原生efcore

首先我们针对原生efcore进行扩展来达到分库分表+code-first自动迁移开发

添加依赖 ShardingCore 6.6.0.3 MySql

//请安装最新版本目前x.6.0.3+,第一个版本号6代表efcore的版本号 Install-Package ShardingCore -Version 6.6.0.3 Install-Package Pomelo.EntityFrameworkCore.MySql -Version 6.0.1 Install-Package Microsoft.EntityFrameworkCore.Tools -Version 6.0.6

创建一个todo实体

public class TodoItem { public string Id { get; set; } public string Text { get; set; } }

创建dbcontext

简单的将对象和数据库做了一下映射当然DbSet+Attribute也是可以的

public class MyDbContext:AbstractShardingDbContext,IShardingTableDbContext { public MyDbContext(DbContextOptions options) : base(options) { } public IRouteTail RouteTail { get; set; } protected override void OnModelCreating(ModelBuilder modelBuilder) { base.OnModelCreating(modelBuilder); modelBuilder.Entity(mb => { mb.HasKey(o => o.Id); mb.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id"); mb.Property(o => o.Text).IsRequired().HasMaxLength(256).HasComment("事情"); mb.ToTable(nameof(TodoItem)); }); } }

新建分库分表路由

分库路由

public class TodoItemDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute { ///  /// id的hashcode取模余3分库 ///  ///  ///  ///  public override string ShardingKeyToDataSourceName(object shardingKey) { if (shardingKey == null) throw new InvalidOperationException("sharding key cant null"); var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString()); return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2 } private readonly List _dataSources = new List() { "ds0", "ds1", "ds2" }; public override List GetAllDataSourceNames() { return _dataSources; } public override bool AddDataSourceName(string dataSourceName) { throw new NotImplementedException(); } ///  /// id分库 ///  ///  public override void Configure(EntityMetadataDataSourceBuilder builder) { builder.ShardingProperty(o => o.Id); } public override Func GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator) { var t = ShardingKeyToDataSourceName(shardingKey); switch (shardingOperator) { case ShardingOperatorEnum.Equal: return tail => tail == t; default: { return tail => true; } } } }

分表路由:

public class TodoItemTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute { public TodoItemTableRoute() : base(2, 3) { } ///  /// 正常情况下不会用内容来做分片键因为作为分片键有个前提就是不会被修改 ///  ///  public override void Configure(EntityMetadataTableBuilder builder) { builder.ShardingProperty(o => o.Text); } }

新建迁移数据库脚本生成

public class ShardingMySqlMigrationsSqlGenerator:MySqlMigrationsSqlGenerator { private readonly IShardingRuntimeContext _shardingRuntimeContext; public ShardingMySqlMigrationsSqlGenerator(MigrationsSqlGeneratorDependencies dependencies, IRelationalAnnotationProvider annotationProvider, IMySqlOptions options,IShardingRuntimeContext shardingRuntimeContext) : base(dependencies, annotationProvider, options) { _shardingRuntimeContext = shardingRuntimeContext; } protected override void Generate( MigrationOperation operation, IModel model, MigrationCommandListBuilder builder) { var oldCmds = builder.GetCommandList().ToList(); base.Generate(operation, model, builder); var newCmds = builder.GetCommandList().ToList(); var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList(); MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds); } }

配置依赖注入

ILoggerFactory efLogger = LoggerFactory.Create(builder => { builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole(); }); var builder = WebApplication.CreateBuilder(args); builder.Services.AddControllers(); builder.Services.AddShardingDbContext() .UseRouteConfig(op => { op.AddShardingTableRoute(); op.AddShardingDataSourceRoute(); }) .UseConfig((sp,op) => { op.UseShardingQuery((con, b) => { b.UseMySql(con, new MySqlServerVersion(new Version())) .UseLoggerFactory(efLogger); }); op.UseShardingTransaction((con, b) => { b.UseMySql(con, new MySqlServerVersion(new Version())) .UseLoggerFactory(efLogger); }); op.AddDefaultDataSource("ds0", "server=127.0.0.1;port=3306;database=mydb0;userid=root;password=root;"); op.AddExtraDataSource(sp=>new Dictionary() { {"ds1", "server=127.0.0.1;port=3306;database=mydb1;userid=root;password=root;"}, {"ds2", "server=127.0.0.1;port=3306;database=mydb2;userid=root;password=root;"} }); op.UseShardingMigrationConfigure(b => { b.ReplaceService(); }); }).AddShardingCore(); var app = builder.Build(); // Configure the HTTP request pipeline. //如果有按时间分片的需要加定时任务否则可以不加 app.Services.UseAutoShardingCreate(); using (var scope = app.Services.CreateScope()) { var defaultShardingDbContext = scope.ServiceProvider.GetRequiredService(); if (defaultShardingDbContext.Database.GetPendingMigrations().Any()) { defaultShardingDbContext.Database.Migrate(); } } //如果需要在启动后扫描是否有表却扫了可以添加这个 //app.Services.UseAutoTryCompensateTable(); //...... app.Run();

添加迁移文件

Add-Migration Init

启动程序

分表分库自动迁移

crud

添加todo字段并迁移

接下来我们将针对TodoItem添加一个name字段并且新增一张既不分库也不分表的表然后进行迁移

public class TodoItem { public string Id { get; set; } public string Text { get; set; } public string Name { get; set; } } public class TodoTest { public string Id { get; set; } public string Test { get; set; } } //docontext protected override void OnModelCreating(ModelBuilder modelBuilder) { base.OnModelCreating(modelBuilder); modelBuilder.Entity(mb => { mb.HasKey(o
                
                

-六神源码网