Flutter Data Structuring

Managing data is not an easy task. As your Flutter app grows in complexity, you’ll need to store and retrieve more data in a structured and efficient way. Fortunately, Firebase offers not only flexible but scalable data storage. You can handle large amounts of data with nearly real-time syncing. In this tutorial, I will share with you the key for structuring data and writing efficient queries such that your Flutter apps can perform really well.

At the end of this tutorial, you should be able to:

• Design a Firestore data structure for scalability.
• Perform advanced Firestore queries (including filtering, ordering, and pagination).
• Optimize Firestore queries for performance.

Let’s start our class.

Designing a Scalable Firestore Data Structure

I always emphasize in my class, a strong foundation or architecture is the beginning of success. Before jumping into data queries, we should first understand how to organize data for scalability. Remember that Firestore is a NoSQL database? That’s mean we stores data in documents and collections instead of traditional rows and tables like SQL databases (MySQL, PostgreSQL, Oracle)

Firestore Data Model

Let us revisit how the data in Firestore are organized

• Documents: Store fields (key-value pairs).
• Collections: Hold multiple documents.

A simple example of a Firestore structure could be:

/users/{userId}/posts/{postId}

Here, a users collection holds documents for each user, and each user has a sub-collection posts where their posts are stored.

Step 2: Performing Advanced Firestore Queries

Firestore provides a powerful query system to help you retrieve data efficiently. Let’s dive into some common and advanced query techniques.

1. Simple Query: Retrieving Documents by Field

Let’s start by retrieving documents based on a specific field, such as getting all tasks marked as complete from a tasks collection.

FirebaseFirestore _firestore = FirebaseFirestore.instance;

Future<void> getCompletedTasks() async {
  QuerySnapshot querySnapshot = await _firestore
      .collection('tasks')
      .where('status', isEqualTo: 'complete')
      .get();

  querySnapshot.docs.forEach((doc) {
    print(doc['taskName']);
  });
}

• where(): Filters documents where the status field is equal to “complete”.
• get(): Executes the query and retrieves the matching documents.

2. Combining Multiple Conditions

You can combine multiple conditions in a query, such as retrieving tasks that are complete and created by a specific user.

Future<void> getUserCompletedTasks(String userId) async {
  QuerySnapshot querySnapshot = await _firestore
      .collection('tasks')
      .where('status', isEqualTo: 'complete')
      .where('userId', isEqualTo: userId)
      .get();

  querySnapshot.docs.forEach((doc) {
    print(doc['taskName']);
  });
}

This query fetches tasks that are both complete and belong to the user with the given userId.

3. Ordering and Limiting Queries

You can also order and limit the number of documents returned by a query. For example, let’s retrieve the most recent tasks:

Future<void> getRecentTasks() async {
  QuerySnapshot querySnapshot = await _firestore
      .collection('tasks')
      .orderBy('createdAt', descending: true)
      .limit(10)
      .get();

  querySnapshot.docs.forEach((doc) {
    print(doc['taskName']);
  });
}

• orderBy(): Orders the documents by the createdAt field in descending order.
• limit(): Limits the number of returned documents to 10.

4. Querying Sub-Collections

Firestore allows you to query sub-collections as well. For example, retrieving all posts from a specific user:

Future<void> getUserPosts(String userId) async {
  QuerySnapshot querySnapshot = await _firestore
      .collection('users')
      .doc(userId)
      .collection('posts')
      .get();

  querySnapshot.docs.forEach((doc) {
    print(doc['postContent']);
  });
}

This query accesses the posts sub-collection for a given userId and retrieves all documents within that sub-collection.

Step 3: Implementing Pagination

When working with large datasets, loading everything at once can cause performance issues. Implementing pagination allows you to load data incrementally.

1. Basic Pagination

To paginate results, use startAfter() or startAt() to load the next batch of documents after the current batch.

DocumentSnapshot? lastDocument;

Future<void> loadMoreTasks() async {
  Query query = _firestore.collection('tasks').orderBy('createdAt').limit(10);

  if (lastDocument != null) {
    query = query.startAfterDocument(lastDocument!);
  }

  QuerySnapshot querySnapshot = await query.get();

  if (querySnapshot.docs.isNotEmpty) {
    lastDocument = querySnapshot.docs.last; // Save the last document for pagination
  }

  querySnapshot.docs.forEach((doc) {
    print(doc['taskName']);
  });
}

• startAfterDocument(): Fetches documents after the lastDocument to load the next page of results.

2. Endless Scrolling with Firestore

To implement infinite scrolling in your app, use a combination of pagination and scroll listeners. This way, as the user scrolls down, new data is automatically loaded.

For instance, in a ListView, you can detect when the user reaches the bottom and then call the loadMoreTasks() function to load more tasks.

Step 4: Optimizing Firestore Queries

While Firestore’s query system is flexible, it’s essential to ensure your queries are optimized, especially as your dataset grows.

1. Indexing

Firestore automatically indexes each field in a collection, but certain complex queries may require composite indexes. If you see an error while running a query, Firebase usually provides a link to create the required index.

For example, a query combining multiple filters:

await _firestore
    .collection('tasks')
    .where('status', isEqualTo: 'complete')
    .orderBy('createdAt', descending: true)
    .get();

This query might require a composite index on the status and createdAt fields. Firebase will prompt you to create this index if it’s needed.

2. Minimizing Read Costs

Firestore charges based on the number of documents you read. To minimize read costs:

• Use pagination to load only the documents you need.
• Use filters (where(), orderBy()) to reduce the number of documents read.
• Avoid deep nesting of sub-collections that require multiple reads.

3. Avoiding Over-Normalization

In NoSQL databases like Firestore, it’s common to denormalize your data to avoid multiple reads. For example, instead of storing user data separately and joining it with tasks, you could include user information directly in each task document. While this increases data redundancy, it reduces the need for complex queries and multiple reads.

Step 5: Real-Time Data with Firestore Listeners

One of Firestore’s most powerful features is real-time syncing, which allows your app to react to changes in the database instantly.

1. Setting Up Real-Time Listeners

You can set up real-time listeners on collections or documents using StreamBuilder to update your app’s UI automatically when data changes.

StreamBuilder<QuerySnapshot>(
  stream: FirebaseFirestore.instance.collection('tasks').snapshots(),
  builder: (context, snapshot) {
    if (snapshot.connectionState == ConnectionState.waiting) {
      return CircularProgressIndicator();
    }
    if (!snapshot.hasData) {
      return Text('No tasks available');
    }

    return ListView(
      children: snapshot.data!.docs.map((doc) {
        return ListTile(title: Text(doc['taskName']));
      }).toList(),
    );
  },
);

This real-time listener automatically updates the UI whenever tasks are added, removed, or modified in Firestore.

Conclusion: Efficiently Querying and Structuring Data in Firestore

You’ve now learned how to design a scalable Firestore data structure, perform advanced queries, implement pagination, and optimize your queries for performance. By leveraging Firestore’s powerful query system and real-time listeners, you can build dynamic apps that handle large datasets with ease.

In the next tutorial, we’ll explore Firebase Cloud Functions to run server-side logic and automate tasks in your Flutter app. Keep coding, and happy building!