Skip to main content

Introduction

Pragmatic, opinionated enhancements to Doctrine's Collections library. Improves the use of Doctrine Collections in large datasets, and other common problems.

Background

We work with huge datasets that are managed by Doctrine ORM, and have complex business rules. These come with these challenges:

  • With standard Doctrine, it seems it is too easy to introduce bugs that will accidentally load the entire dataset into memory and cause out-of-memory errors. And these sorts of errors will usually only show up in production, but never in the development environment.

  • Iterating over large datasets with the correct method is difficult and cumbersome. You usually need to devise custom solutions for each use case.

  • Counting the number of records is very slow. Sometimes we can do away with the count, sometimes it is a must, and we need to work around the problem.

Other, non-performance issues include:

  • Doctrine's Selectable appears to be a prevalent abstraction leak. Coders tend to litter the codebase with internal-revealing Criteria objects, and updating the entity can potentially become a nightmare. No static analysis tool can currently detect this problem. In fact, some exacerbate the problem by assuming a Collection must also be a Selectable.

Previously, we created rekalogika/doctrine-collections-decorator to solve these problems. However, it is still too cumbersome because we need to approach the problem one at a time. We need a more comprehensive solution.

Components

  • Decorator classes that enhance any Doctrine Collections classes.
  • Query-backed collections. Turns a QueryBuilder into a lazy-loading collection.
  • An alternative implementation of the repository pattern that implements Collection.
  • Modifications to ArrayCollection that does matching() against the private properties directly, to reproduce the same behavior of PersistentCollection.

Features

  • Safeguards against potential out-of-memory situations. Throws an exception before it hits harder-to-debug out-of-memory situation.
  • Pluggable counting strategies. Work around long counting times using your own counting strategies.
  • Full versions of the collection classes that offer full compatibility with the original Doctrine Collection. And the minimal flavors that only expose the safe methods.
  • Built in keyset pagination using rekalogika/rekapager library. Iterate over collections of any size without loading them all into memory. And without having to create ad-hoc queries every time you need to achieve that.
  • Option to use the traditional offset pagination instead of keyset pagination.
  • Keyset pagination can also be used to create pagination for user interfaces and API outputs.
  • Encourages you to create expressive, higher-level methods to provide the same functionality as the Selectable interface, but without exposing the inner workings of the class.

License

MIT

Contributing

Issues and pull requests should be filed in the GitHub repository rekalogika/collections.