In today's world, data is invaluable for businesses and individuals, describing the real world. In digital terms, we’re seeing more data than ever before, thanks to technologies like AI, IoT, blockchain, etc.. Even for small and simple apps you need solid data modeling to run smoothly and grow easily. The proper usage of data is crucial for creating systems and applications that improve our lives daily. However, fast management of large amounts of diverse data poses challenges that are not easy to solve. Database paradigms help us overcome these issues in amazing ways.

Let us start with defining and explaining elemental concepts:

• Data: we can mathematically define data as an ordered quadruple (e, p, v, t), where it represents the specific value (v) of a specific property (p) for a given entity (e) at a certain point in time (t).
• Database: which represents a collection of data
• Database paradigm: a fundamental approach for organizing, storing, and manipulating data within a database system**.**

The different ways of organizing and manipulating data can help us to be performant in different use case scenarios, so understanding when and how to use them is important for designing scalable, efficient, and maintainable systems.

Let’s hop on this journey starting from the most common one:

Relational databases, or as a lot of people love to call them SQL databases, have become the backbone for most of the data management systems today. They are designed to follow a structured approach to organizing and handling data in such a way that it becomes imperative to store and retrieve information in an organized way. Unlike other database types, relational databases store their data in tabular format, a collection of rows and columns. This is a form in which we can store information efficiently since each row contains a unique record and each column defines an attribute or characteristic for the data. Used in such a way it is made very easy to visualize and understand for the users. And for most of your apps, this is just enough, it will work every time!

But why does it work so well? The background of the relational database is some (not so easy) math which we won’t dive into now. It is based on a theoretical framework, relational algebra, developed in the 1970s. The latter encompasses some operations such as selection, projection, union, and join to operate on the relationships of data. A relational database, based on this principle of relational algebra, maintains the integrity of the data and enforces consistency with good efficiency during all retrieval and manipulation operations.

Relational databases are quite widespread in the industry, having very strong concepts of reliability, scalability, and flexibility. Their functionalities for maintaining structured data are outstanding, supporting transactions, concurrency control, and integrity constraints on data. The popular RDBMS includes MySQL, PostgreSQL, Oracle, SQL Server, and SQLite each offering different capabilities and performance characteristics.

The paragraph above mentions a lot of SQL. It represents the Structured Query Language. This is a well-known, well-documented language used for querying and manipulating the data within the relational database. It supports various types of commands like Data Definition Language (DDL), Data Manipulation Language (DML), Data Control Language (DCL), Data Query Language (DQL), and Transaction Control Language (TCL).

• Tables: The data is stored in tables, which are two-dimensional structures consisting of rows and columns. Each table models a specific entity or concept, and each row represents a unique instance of that entity, while each column represents a specific attribute or property of the entity. Note: Here we can once again go to the definition of data and see why this approach is so good at storing it.
• Primary Key: a column or set of columns that uniquely identifies each row in a table. It is used as an ID of that entity, it enforces entity integrity and ensures that if we query for a specific ID we must always get a single specific instance of an entity! This property is indexed by default which enables fast retrieval of data by this property. There are a lot of types of indices (unique, composite, full text, etc.) and all columns of the table can be indexed but the primary key is by default.
• Foreign Key: a column or set of columns in one table that refers to the primary key in another table. It establishes relationships between tables and enforces referential integrity, ensuring that the values in the foreign key column(s) correspond to valid values in the primary key column(s) of the related table. Through joins commands we can retrieve a lot of mutually connected data
• Normalization: is the process of organizing data in a way to minimize redundancy and dependency by dividing large tables into smaller, related tables and defining relationships between them. Normalization helps us to improve data integrity, reduce data duplication, and simplify database maintenance.
• ACID Transactions: Relational databases are ACID compliant (Atomicity, Consistency, Isolation, Durability) which is one of their key characteristics. This means that transactions (which we can define as every action done on our data) are treated as indivisible units (A), ensuring that either all operations within a transaction will be completed or all will fail, thus preventing the loss of data integrity.

BEGIN

UPDATE bank_accounts
SET balance = balance - :amount
WHERE account_number = :yourBankAccount;

COMMIT;
Copy

BEGIN

UPDATE bank_accounts
SET balance = balance + :amount
WHERE account_number = :myBankAccount;

COMMIT;
Copy

async function transferMoneyNoAtomicity(
  amount: number,
  yourBankAccount: string,
  myBankAccount: string
) {
  await db.subtractMoney(amount, yourBankAccount)
  await db.addMoney(amount, myBankAccount)
}

Copy

The example from above is a simple way of presenting a bad implementation. If the first action of subractMoney() worked fine, the money would successfully be removed from your account and that change would be saved. But if the second action of addMoney() failed, I would not get the money as intended, and you would not get your money back either! That is not the situation we want to be in, neither as devs who implemented this function, banks who provide the service nor one of those two users who are involved in transactions. To fix this issue we can simply do something like:

BEGIN;

UPDATE bank_accounts
SET balance = balance - :amount
WHERE account_number = :yourBankAccount;

UPDATE bank_accounts
SET balance = balance + :amount
WHERE account_number = :myBankAccount;

COMMIT;

Copy

async function transferMoneyWithAtomicTransaction(
  amount: number,
  yourBankAccount: string,
  myBankAccount: string
) {
  await db.wrapInTransaction(async () => {
    await db.subtractMoney(amount, yourBankAccount)
    await db.addMoney(amount, myBankAccount)
  })
}
Copy

wrapInTransaction() is a function method (usually already provided within various Object Relation Mappers or ORMs ) that provides a common context for the inner functions that do connect to a database. Now, if the addMoney() fails, both update actions will be rollbacked and you guys won’t lose any money!

Relational databases maintain the consistency of data (C) by guaranteeing that the database remains in a valid state before and after each transaction. Moreover, they provide isolation (I), ensuring that transactions operate independently of each other to prevent interference, and durability (D), guaranteeing that committed transactions will persist even in the event of system failures, thus safeguarding against data loss.

So even if developed a long time ago the mathematical concepts behind it, easy storing data within tables, consistency assurance, and a well-defined language such as SQL make relational databases still the best and most used database form. If the problem you’re encountering is not specific for performance or domain reasons, the suggestion would be to always go (or at least think about it first) with using one of these databases. They are good for most use cases, but they have some bad sides, such as speed, relations, and inability to handle unstructured data (which is very common in today's world) don’t worry, others cover their bad sides.

Key-value databases represent a fundamental departure from the traditional relational model, offering a highly flexible and scalable approach to data storage. In these databases, data is organized as a collection of key-value pairs, where each key uniquely identifies a corresponding value. This simplistic but powerful data model allows for efficient storage and retrieval of information without the constraints of predefined schemas that we found as an issue in relational databases.

As a developer working with schema-less or schema-flexible data format means that you are not required to define a fixed schema upfront, allowing for dynamic changes to the data model as application requirements evolve. This flexibility is particularly advantageous for handling semi-structured or unstructured data, where the schema may vary between different entities or data points.

Another key advantage of key-value databases is their high scalability and fast retrieval of data, which is what we need in commercial-ready solutions. These databases are designed to handle massive volumes of data with high throughput by distributing data and processes usually on a single thread, with an event-driven architecture. For the heavier workload solutions, most databases of this kind have a Cluster solution, and the vertical scalability scales with the power of the machine.

The most popular example of a key-value database is Redis. Redis is well known for its exceptional performance and versatility, offering features such as in-memory caching, data persistence, and support for various data structures like strings, lists, sets, and sorted sets. It stores data in RAM, which saves us from waiting for the data to go all the way to the machine disk or SSD, and looks for it, which takes a lot of time. It is primarily used for the cache layer of data in front of your database, but it can also be used as a legitimate database with the possibility to persist the data on the disc while storing a replica on the RAM which maintains the fast retrieval of data. It utilizes the event loop (in a similar way as JavaScript does) for handling huge amounts of requests despite being a single-threaded

Redis in particular is really easy to use despite the “complicated” architecture in the background

• Key-Value Pairs: The fundamental unit of data storage in key-value databases a simple model that is easy to understand. Each key is unique and maps to a corresponding value. Keys are used to retrieve and manipulate associated values efficiently. The value of data can be as simple as strings but can be more complex like sets, lists, geolocation data, etc…
• Schema-less: Key-value databases typically have a schema-less or schema-flexible structure, allowing for dynamic changes to the data model without requiring a predefined schema. This flexibility is well-suited for handling semi-structured or unstructured data.
• Fast Retrieval: Key-value databases offer fast and efficient retrieval of data based on keys using a hash function and data structures like B-tree, skip list, etc. By directly accessing data via its key, key-value databases can achieve low-latency read and write operations, making them suitable for use cases requiring real-time data access. Often we do not set our key-value database on our machine but in RAM, so we save time with accessing data.
• High Scalability: Key-value databases are designed for high scalability and performance. They can easily scale horizontally by distributing data across multiple nodes in a cluster, utilizing single-thread and fast lookups: O(1) on the RAM making them ideal for handling large volumes of data and high throughput workloads.

In general, key-value databases are flexible, scalable, and efficient means for storing and managing data in the most modern applications. These databases, embracing simplicity with a powerful data model, give the developer the capability to create resilient and adaptable systems that can respond with ease to differing data requirements.

Column-wide databases, also known as column-family stores or wide-column stores, are rare ones in the NoSQL ecosystem in that, instead of holding data in rows, they store it vertically, grouping similar data into columns. This basic architectural change enables column-wide databases to scale and adapts in ways that their row-based siblings cannot, efficiently supporting high-volume datasets with varied structures and access patterns.

They provide a flexible architecture in which evolving data requirements can easily be fulfilled without giving away performance. Utilizing a columnar data model, each column is stored sequentially on disk and can be independently accessed and manipulated, combined with advanced indexing, column-wide databases excel in high-throughput real-time analytics and dynamic schema evolution.

Apache Cassandra and Apache HBase continue to be two of the most recognizable implementations of column-wide databases and are widely recognized for their robustness, scalability, and fault tolerance. Among them, Cassandra is widely recognized with high-level acknowledgment for its decentralized architecture, linear scalability, and tunable consistency levels, and is considered a popular choice for mission-critical applications in various industries. Its query language, CQL, is very close to SQL, so it is rather easy to use and migrate from Relational to Columnar database. Unlike the relational database, it can’t do joins, but it is much easier to scale data on multiple nodes and replicate it.

Another advantage of these kinds of databases is that they store data in batches. Firstly, it stores it in memory, and after some time it just saves all of that on the disk, which makes this kind of database perfect for data that requires frequent writes, but not so frequent reads and updates, like in situations from IoT devices, weather sensors, etc.

• Column Families: In column-wide databases, data is organized into column families, which are logical groupings of columns. Each column family can contain an arbitrary number of columns, and each column can hold a single value associated with a row key.
• Rows and Columns: Data is stored in rows, with each row identified by a unique row key. Unlike traditional relational databases where rows are of fixed structure, rows in column-wide databases can have varying numbers of columns, allowing for flexible schema design.
• Column-Oriented Storage: Unlike row-oriented databases where data for each row is stored contiguously, column-wide databases store data for each column contiguously on disk. This column-oriented storage approach enables efficient read and write operations, especially for analytics and data warehousing use cases.
• Column-Level Indexing: Column-wide databases support indexing at the column level, enabling efficient querying and retrieval of data based on specific columns. This indexing capability enhances query performance, especially for analytical queries that involve aggregations and filtering.

In all, column-wide databases are a paradigm shift in data management, offering an unprecedented combination of scalability, flexibility, and performance required by today's demanding data-driven applications.

Document-oriented databases, a dominant category within the NoSQL family, revolutionize data storage by solving a flexible and schema-less approach. At the core of these databases lies the concept of documents (a more dynamic and adaptable model), which serve as the fundamental unit of data storage in this case. They are pretty easy to imagine since you can think of them like suitcases with a bunch of papers in them, where those papers represent the data. But remember the papers can be different from one another!

In document-oriented databases like MongoDB, data is stored, retrieved, and managed in the form of JSON-like documents. These documents encapsulate information in a self-contained way, allowing for nested structures, arrays, and key-value pairs but avoiding things like joins and foreign keys. Such flexibility enables developers to represent complex data structures without the constraints imposed by strict schemas.

Document-oriented databases usually have powerful query languages or APIs that can run complex operations on them like filtering, sorting, and aggregation, including even nested queries. For instance, MongoDB provides a rich query language and an aggregation framework that allows the developer to do complex data manipulation tasks straightforwardly.

One of the key advantages of document-oriented databases is their schema flexibility. Unlike relational databases, where schema changes often require changes through migration processes, document-oriented databases allow developers to easily change their data models. This agility is very helpful in cases where the schema of the data changes often. We could even have a situation where the different instances of the same kind of documents could have properties that are different from one another, which in many cases is useful and forbidden inside relational databases.

{
  "_id": 1,
  "first_name": "Tom",
  "email": "tom@example.com",
  "cell": "765-555-5555",
  "likes": [
    "fashion",
    "spas",
    "shopping"
  ],
  "businesses": [
    {
      "name": "Entertainment 1080",
      "partner": "Jean",
      "status": "Bankrupt",
      "date_founded": {
        "$date": "2012-05-19T04:00:00Z"
      }
    },
    {
      "name": "Swag for Tweens",
      "date_founded": {
        "$date": "2012-11-01T04:00:00Z"
      }
    }
  ]
}

Copy

{
  "_id": 2,
  "first_name": "Donna",
  "email": "donna@example.com",
  "spouse": "Joe",
  "likes": [
    "spas",
    "shopping",
    "live tweeting"
  ],
  "businesses": [
    {
      "name": "Castle Realty",
      "status": "Thriving",
      "date_founded": {
        "$date": "2013-11-21T04:00:00Z"
      }
    }
  ]
}

Copy

As you can see, those two records in the database have some common characteristics, but are diverse in the property naming with cell and spouse. You can think of this as a Users table in the relational database, but it has different columns based on the record. One more important thing to notice is the _id property in both records, which is MongoDB's autogenerated internal id, which is used for indexing and fast retrieval.

Another signature of document-oriented databases is scalability. They are designed to be scaled horizontally, which distributes the data over nodes or servers. This kind of distributed architecture makes them capable of handling large volumes of data and high throughput workloads, hence being fit for applications whose data size is increasing with time.

• Document: The fundamental unit of data storage in document-oriented databases. Each document is a self-contained data structure, typically represented in formats like JSON, BSON, and XML. Documents can contain nested structures, arrays, and key-value pairs, providing flexibility in data representation.
• Schema Flexibility: Document-oriented databases offer schema flexibility, allowing for dynamic changes to the data model without requiring a predefined schema. This flexibility is well-suited for handling semi-structured or unstructured data, as documents within the same collection can have different structures.
• Querying: Document-oriented databases provide powerful querying capabilities, enabling complex queries on nested document structures and dynamic schema. Query languages or APIs are used to perform operations such as filtering, sorting, aggregation, and even nested queries.

Above all, document-oriented databases provide a powerful alternative to traditional relational databases. Developers can leverage flexible, scalable, and agile ways of building up-to-date applications driven basically by data. By embracing a schema-free approach and pushing toward document-centered data models, these databases support developers in innovating and iterating quickly given dynamic enterprise requirements.

Graph databases provide a different approach to data storage by representing and storing data in the form of graph structures. In graph databases, data is modeled as nodes and edges to accurately represent any relationship or dependency in the real world. Nodes represent entities such as people, products, or places, while edges represent the connections between those entities. This is a very natural, intuitive way to store complex and interconnected data. Each node and edge of a graph may have properties, which include further information and metadata on the entities and their relationships: things like names, age, weights, or whatever else that allows for more detailed querying and analysis.

Graph databases are ideal for cases where the relationships that interlink data items are as important as the data itself. They are particularly suited for complex querying, pattern matching, and the traversal of interrelated data in applications such as social networks, recommendation systems, fraud detection, network studies, and knowledge graphs. Compared to relational databases where we would have to join multiple tables to collect all needed information, here we can just find a specific relationship of a specific entity and retrieve all that it is connected to, which is a much cheaper operation.

Many graph databases have risen to the top in the market where Neo4j takes the lead, providing a robust and feature-rich platform to build graph-powered applications. Amazon Neptune is a fully managed graph database service by AWS that offers scalability, reliability, and frictionless integration with other AWS services. Query languages like Cypres make querying on this kind of database very intuitive since all you have to do is follow the certain relations for the info you want to get:

MATCH (m:Movie {title: 'The Dark Knight'})
OPTIONAL MATCH (a:Actor)-[:ACTED_IN]->(m)
OPTIONAL MATCH (d:Director)-[:DIRECTED]->(m)
RETURN m.title AS Movie, m.released AS ReleaseYear,
       collect(DISTINCT a.name) AS Actors,
       collect(DISTINCT d.name) AS Directors
Copy

• Nodes: Nodes are the fundamental units of data storage in graph databases. They represent entities such as people, products, or locations. Each node can have properties that describe the characteristics or attributes of the entity it represents.
• Edges: Edges, also known as relationships or edges, represent the connections between nodes. They define the relationships between entities in the graph. Like nodes, edges can also have properties that describe additional information about the relationship.
• Properties: Properties are key-value pairs associated with nodes and edges. They provide additional information about the nodes and edges in the graph. Properties can be used to store attributes such as names, ages, or weights within the nodes or edges themselves.
• Graph Theory: Graph theory is a complex mathematical part with a lot of fast and well-thought-out algorithms for quick searching, sorting, and managing nodes and edges. Therefore, basic knowledge of graph theory is very important for efficient and professional usage of these databases.

Graph databases represent a powerful paradigm in the ways data connected to other data are represented and queried. Properties in graph structures permit intuitive modeling of complex relationships and the capability to traverse such graphs efficiently is central for an entire spectrum of application fields across most domains.

The evolution of technology continually drives and creates more new paradigms and enhancements to existing ones. For instance, the advancement of AI made emerging paradigms like vector databases introduce novel approaches for managing high-dimensional data efficiently, catering to the needs of modern data-intensive applications. NewSQL is a class of modern relational databases (often used by big companies to solve their data problems), management systems (RDBMS) that combine the scalability of NoSQL systems with the ACID compliance of traditional SQL databases. These systems aim to provide high-performance, fault-tolerant, and horizontally scalable solutions for handling large-scale transactional workloads… The search-based databases like ElasticSearch can index huge amounts of text and perform fast lookups for specific words on a large amount of data with the compensation of compute power. For storing some application events, logs, or service metrics, a time-series database like InfluxDB can be very helpful. These are just some of many new approaches, and yet many new ones will probably appear.

Database paradigms represent a variety of data management strategies, each specified with different challenges and requirements. From the flexible and scalable nature of key-value and document-oriented databases to the structured integrity of relational databases and the relationship-driven approach of graph databases, a wide array of paradigms exists to suit diverse application needs.

In this ever-evolving ecosystem, database technologies are continuously improving, offering developers increasingly powerful tools to handle complex data scenarios. By keeping up with these advancements and understanding the strengths and weaknesses of each paradigm, developers can make big decisions to architect systems that not only meet current demands but also adapt to future challenges and opportunities in every problem they encounter.

In the end, I would like to show you my cheatsheet for using those paradigms:

To find out more, you can check on additional resources:

• https://www.ibm.com/think/topics/relational-databases

• https://cassandra.apache.org/doc/latest/cassandra/getting-started

• https://www.mongodb.com/resources/basics/databases/document-databases

• https://redis.io/docs/latest/

• https://neo4j.com/docs/getting-started/graph-database/

⁤See you in our next blog with a new topic!