Creating a data layer in Flutter

ℹ There is a new version of this article in which I explain the same thing but I make use of more plugins to reduce the development time. You can see it here.

In this article I am going to explain how to create a data layer in a Flutter app. The objective is to be able to manage the input/output of data in our application in the most efficient way, using in this case two data sources: a remote API and a local database.

The pattern that I am going to explain here is an implementation of the Repository Pattern, typically included within the scope of the Clean Architecture. Although this system admits several interpretations, the implementation that I have been using for quite some time and that has worked quite well for me defines two basic elements:

Data source: this component performs raw operations against the data, an example would be a class dedicated to making calls against an API, or a Data Access Object (DAO) that performs queries against a database.
Repository: this component articulates one or more data sources, and decides where the data should flow to or from.

On this last point, and for better understanding, I will explain it on the example that I am going to define below:

The objective is to show the user a list of data, for this example I will use the CoinCap API, from which I will obtain a list of cryptocurrencies.
What I want is for the data to be saved locally in such a way that if the user re-enters the app it will load it locally instead of fetching it from the remote API.
To do this, I will first create a class whose function will be to consume the CoinCap API and obtain said data.
I will also create a class to perform queries against a local database.
These two classes will be controlled by a repository object, in such a way that the app will only work with this data through this class.
In the first call, the repository object will look if there is data in the database and if it exists it will return it, if it does not exist it will look for it in the API and then it will save it in the database.

Note: If you are one of those who prefers to learn by observing code, here is the complete project.

Defining our domain models

The first step is to define the data structures that we are going to need. These data should not be conditioned by the way in which we are going to receive it from the data sources, although in many cases they are similar.

For this example I am going to define my Asset object, which represents a given cryptocurrency with its name, current value, etc:

class Asset {
  final String id;
  final String symbol;
  final String name;
  final double? supply;
  final double? maxSupply;
  final double priceUsd;

  Asset({
    required this.id,
    required this.symbol,
    required this.name,
    this.supply,
    this.maxSupply,
    required this.priceUsd,
  });
}

Creating the network layer

Now we are going to create the necessary classes to be able to work with the remote API. To do this, first we are going to define an entity that must be a faithful representation of the model that we receive from the API, in this case I am going to omit some fields that do not interest me:

Note: The official documentation explains very well how to work with a remote API.

class AssetsResponse {
  final List<AssetEntity> data;

  AssetsResponse({required this.data});

  factory AssetsResponse.fromJson(Map<String, dynamic> json) {
    final data = json['data'] as List<dynamic>;

    final List<AssetEntity> entities = [];
    for (final element in data) {
      entities.add(AssetEntity.fromJson(element as Map<String, dynamic>));
    }

    return AssetsResponse(data: entities);
  }
}

class AssetEntity {
  final String id;
  final String symbol;
  final String name;
  final String? supply;
  final String? maxSupply;
  final String priceUsd;

  AssetEntity({
    required this.id,
    required this.symbol,
    required this.name,
    this.supply,
    this.maxSupply,
    required this.priceUsd,
  });

  AssetEntity.fromJson(Map<String, dynamic> json)
      : id = json['id'] as String,
        symbol = json['symbol'] as String,
        name = json['name'] as String,
        supply = json['supply'] as String?,
        maxSupply = json['maxSupply'] as String?,
        priceUsd = json['priceUsd'] as String;
}

As you can see, in this case this entity object has the same fields as the domain object. There are ways to simplify these processes using a single class, however I prefer to have two separate ones. This gives me the flexibility to modify both parts independently when necessary, if we don't do this we would be creating a coupling of a data source with our business logic, and this can be counterproductive as our application grows and its complexity increases.

Now all I have to do is create the API client, which I am going to extend from a class with network functionalities for general use:

class ApiClient extends BaseClient {
  final String baseUrl;

  ApiClient({
    required Logger log,
    required this.baseUrl,
  }) : super(log: log);

  Future<AssetsResponse> getAssets() async {
    final response = await get(
      '${baseUrl}assets',
      headers: {'Accept-Encoding': 'gzip'},
    );

    // Simulate a network delay
    await Future.delayed(const Duration(seconds: 3));

    return AssetsResponse.fromJson(
      jsonDecode(response.body) as Map<String, dynamic>,
    );
  }
}

abstract class BaseClient {
  static const _timeout = 30000;
  static const _retries = 1;

  final Logger log;
  late final http.Client _client;

  BaseClient({required this.log}) {
    _client = http.Client();
  }

  @protected
  Future<http.Response> get(
    String url, {
    Map<String, String>? headers,
    int? currentTry,
    Map<String, dynamic>? queryParameters,
  }) async {
    int retry = currentTry ?? 0;

    String queryParams = '';
    if (queryParameters != null) {
      queryParams += '?';
      queryParameters.forEach((key, value) {
        queryParams += '$key=$value&';
      });
      queryParams = queryParams.substring(0, queryParams.length - 1);
    }

    try {
      final uri = Uri.parse('$url$queryParams');
      return await _client
          .get(uri, headers: headers)
          .timeout(const Duration(milliseconds: _timeout));
    } on TimeoutException catch (_) {
      log.w("Timeout after $_timeout milliseconds:");
      log.w("-- URI: $url");

      if (retry < _retries) {
        retry++;
        return get(url, headers: headers, currentTry: retry);
      } else {
        rethrow;
      }
    } catch (e) {
      rethrow;
    }
  }
}

Note: I am simulating a 3 seconds delay in the network request. I do this so that one can easily see when the app is getting the data from the API, and when it's getting it from the local database. If you run the test project you can see it for yourself.

Creating the database layer

Now we are going to carry out the same process, but this time working against a local database. First I am going to create the representation of an entry in the assets table:

Note: Again, the official docs explain very well how to work with a local sqlite database.

class AssetDbEntity {
  static const fieldId = 'id';
  static const fieldSymbol = 'symbol';
  static const fieldName = 'name';
  static const fieldSupply = 'supply';
  static const fieldMaxSupply = 'max_supply';
  static const fieldPriceUsd = 'price_usd';

  final String id;
  final String symbol;
  final String name;
  final double? supply;
  final double? maxSupply;
  final double priceUsd;

  AssetDbEntity({
    required this.id,
    required this.symbol,
    required this.name,
    this.supply,
    this.maxSupply,
    required this.priceUsd,
  });

  AssetDbEntity.fromMap(Map<String, dynamic> map)
      : id = map[fieldId] as String,
        symbol = map[fieldSymbol] as String,
        name = map[fieldName] as String,
        supply = map[fieldSupply] as double?,
        maxSupply = map[fieldMaxSupply] as double?,
        priceUsd = map[fieldPriceUsd] as double;

  Map<String, dynamic> toMap() => {
        fieldId: id,
        fieldSymbol: symbol,
        fieldName: name,
        fieldSupply: supply,
        fieldMaxSupply: maxSupply,
        fieldPriceUsd: priceUsd,
      };
}

Now I create a Data Access Object (DAO) that accesses the data, which in turn extends from a generic class:

class AssetDao extends BaseDao {
  Future<List<AssetDbEntity>> selectAll() async {
    final db = await getDatabase();
    final List<Map<String, dynamic>> maps =
        await db.query(BaseDao.assetTableName);
    return List.generate(maps.length, (i) => AssetDbEntity.fromMap(maps[i]));
  }

  Future<void> insertAll(List<AssetDbEntity> assets) async {
    final db = await getDatabase();
    final batch = db.batch();

    for (final asset in assets) {
      batch.insert(BaseDao.assetTableName, asset.toMap());
    }

    await batch.commit();
  }
}

abstract class BaseDao {
  static const databaseName = 'data-layer-sample.db';

  static const assetTableName = 'asset';

  @protected
  Future<Database> getDatabase() async {
    return openDatabase(
      join(await getDatabasesPath(), databaseName),
      onCreate: (db, version) async {
        final batch = db.batch();
        _createAssetTable(batch);
        await batch.commit();
      },
      version: 1,
    );
  }

  void _createAssetTable(Batch batch) {
    batch.execute(
      '''
      CREATE TABLE $assetTableName(
      ${AssetDbEntity.fieldId} TEXT PRIMARY KEY NOT NULL,
      ${AssetDbEntity.fieldSymbol} TEXT NOT NULL,
      ${AssetDbEntity.fieldName} TEXT NOT NULL,
      ${AssetDbEntity.fieldSupply} REAL,
      ${AssetDbEntity.fieldMaxSupply} REAL,
      ${AssetDbEntity.fieldPriceUsd} REAL NOT NULL
      );
      ''',
    );
  }
}

Class mapping

Before creating our repository, we need a class that performs model conversions:

class MapperException<From, To> implements Exception {
  final String message;

  const MapperException(this.message);

  @override
  String toString() {
    return 'Error when mapping class $From to $To: $message}';
  }
}

class Mapper {
  Asset toAsset(AssetEntity entity) {
    try {
      return Asset(
        id: entity.id,
        symbol: entity.symbol,
        name: entity.name,
        supply: entity.supply != null ? double.parse(entity.supply!) : null,
        maxSupply:
            entity.maxSupply != null ? double.parse(entity.maxSupply!) : null,
        priceUsd: double.parse(entity.priceUsd),
      );
    } catch (e) {
      throw MapperException<AssetEntity, Asset>(e.toString());
    }
  }

  List<Asset> toAssets(List<AssetEntity> entities) {
    final List<Asset> assets = [];

    for (final entity in entities) {
      assets.add(toAsset(entity));
    }

    return assets;
  }

  Asset toAssetFromDb(AssetDbEntity entity) {
    try {
      return Asset(
        id: entity.id,
        symbol: entity.symbol,
        name: entity.name,
        supply: entity.supply,
        maxSupply: entity.maxSupply,
        priceUsd: entity.priceUsd,
      );
    } catch (e) {
      throw MapperException<AssetDbEntity, Asset>(e.toString());
    }
  }

  List<Asset> toAssetsFromDb(List<AssetDbEntity> entities) {
    final List<Asset> assets = [];

    for (final entity in entities) {
      assets.add(toAssetFromDb(entity));
    }

    return assets;
  }

  AssetDbEntity toAssetDbEntity(Asset asset) {
    try {
      return AssetDbEntity(
        id: asset.id,
        symbol: asset.symbol,
        name: asset.name,
        supply: asset.supply,
        maxSupply: asset.maxSupply,
        priceUsd: asset.priceUsd,
      );
    } catch (e) {
      throw MapperException<Asset, AssetDbEntity>(e.toString());
    }
  }

  List<AssetDbEntity> toAssetsDbEntity(List<Asset> assets) {
    final List<AssetDbEntity> list = [];

    for (final asset in assets) {
      list.add(toAssetDbEntity(asset));
    }

    return list;
  }
}

Creating the repository

To finish, we are going to define a repository centered on the Asset object that is going to perform all the data logic that I have described before:

class AssetRepository {
  final ApiClient apiClient;
  final Mapper mapper;
  final AssetDao assetDao;

  AssetRepository({
    required this.apiClient,
    required this.mapper,
    required this.assetDao,
  });

  Future<List<Asset>> getAssets() async {
    // First, try to fetch the assets from database
    final dbEntities = await assetDao.selectAll();

    if (dbEntities.isNotEmpty) {
      return mapper.toAssetsFromDb(dbEntities);
    }

    // if the database is empty, fetch from the API, saved it to database,
    // and return it to the caller
    final response = await apiClient.getAssets();
    final assets = mapper.toAssets(response.data);

    await assetDao.insertAll(mapper.toAssetsDbEntity(assets));

    return assets;
  }
}

Using our repository to display data

Now we just need to use this repository in our presentation layer to display the data:

class MainScreen extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text('Data layer sample')),
      body: FutureBuilder<List<Asset>>(
        future: Provider.of<AssetRepository>(context).getAssets(),
        builder: (BuildContext context, AsyncSnapshot<List<Asset>> snapshot) {
          if (snapshot.hasData) {
            return ListView.separated(
              itemBuilder: (context, index) {
                final asset = snapshot.data![index];
                final StringBuffer buffer = StringBuffer();

                if (asset.supply != null) {
                  buffer.write('Supply: ${asset.supply!.toStringAsFixed(2)}');

                  if (asset.maxSupply != null) {
                    buffer.write(
                      ' / Max supply: ${asset.maxSupply!.toStringAsFixed(2)}',
                    );
                  }
                }

                return ListTile(
                  leading: Text(
                    asset.symbol,
                    style: const TextStyle(
                      color: Colors.black54,
                      fontWeight: FontWeight.bold,
                    ),
                  ),
                  title: Text(asset.name),
                  subtitle: Text(buffer.toString()),
                  trailing: Text('\$${asset.priceUsd.toStringAsFixed(2)}'),
                );
              },
              separatorBuilder: (context, index) => const Divider(),
              itemCount: snapshot.data!.length,
            );
          } else if (snapshot.hasError) {
            return const Center(
              child: Text('An error occurred while fetching data.'),
            );
          } else {
            return const Center(
              child: CircularProgressIndicator(),
            );
          }
        },
      ),
    );
  }
}

Conclusion

With this we already have everything we need. I am aware that there are quite a few classes and quite a bit of code to carry out this process, and as I said before, there are other simpler and more beginner-friendly methodologies. However, I believe that getting used to working with well-defined boundaries and determining precisely which layer each thing belongs to will help us in the future to be able to manage larger volumes of data much more efficiently.

You can see the source code of this article here.

Thanks for reading this far and happy coding!

The cover image is derivated from a photo by Pietro Jeng on Unsplash

Creating a data layer in Flutter

Comments

More from this blog

My (Dark) Prediction for Android

🛠️ How to Use Multiple Windows in Flutter Desktop

5 Tips to Optimize Your Flutter App 🔥🚀

Your home address exposed on Google Play 😰

Learn Flutter by creating your first Flutter app!

Defining our domain models

Creating the network layer

Creating the database layer

Class mapping

Creating the repository

Using our repository to display data

Conclusion

Command Palette

Comments

More from this blog

Defining our domain models

Creating the network layer

Creating the database layer

Class mapping

Creating the repository

Using our repository to display data

Conclusion