We all have to do them, so do them well
Code review is just as important as writing code
If code/pull request (PR) reviews aren’t timely, it’s costly to team productivity. Also, the longer a PR languishes, the more it gets out of sync with the target branch and thus harder to merge. And then everyone’s motivation to review/merge diminishes.
The clear lesson: stay on top of your code reviews. My personal rule of thumb is that all PRs should get an initial review within 24 hours of being created.
Make smaller PRs
Smaller PRs are inherently virtuous for a number of reasons:
- They’re less taxing on the reviewer
- They’re more likely…
Update, 12/7/21: Amazon has apparently enabled strong consistency for S3 by default, meaning this article’s description of S3 as eventually consistent is no longer valid and the techniques described for handling it should no longer be necessary. However, feel free to read on if you want a better understanding of eventually consistent systems in general or if you want a better appreciation of what Amazon making S3 strongly consistent really means.
Background
Eventual consistency is one of those computer science words that I didn’t learn in college. But my basic understanding of it is that it describes when data writes are…
Example data: Cumulus granules/files data
Recently on Cumulus, we wanted to re-examine how we’re storing our granules/files data. For context, on Cumulus a granule is a set of related data science files, so granules are composed of files. The granules and files are stored in separate tables. Granules to files have a one-to-many relationship.
Cumulus is currently using DynamoDB to store this data, which is a NoSQL database, and as a result there is no referential integrity between the file records and the granule record they belong to. …
I’m excited to welcome Jennifer to Development Seed! Jennifer is a Cloud Engineer who is passionate about accessibility and meaningfully expressing systems and concepts in code.
Alongside the Earthdata team, she will be working to build core infrastructure for Cumulus, which helps to better leverage cloud computing for data processing, storage, and retrieval of NASA’s Earth observation data.
Before joining Development Seed, Jennifer worked as a Junior Software Engineer at Acorns where she helped build the backend of the Acorns Spend product.
When not contributing to open source work, Jennifer enjoys meditating and trying new recipes.
I’m thrilled to welcome Chuck Daniels to both the Development Seed and Cumulus teams! Chuck is a skilled data engineer and programmer and adds some serious technical chops to our infrastructure team. He’ll help expand our cloud infrastructure capabilities for data providers like NASA.
Chuck and I will work closely together on the Cumulus project, building towards a future of cloud-based scientific analysis by developing software for configurable data pipelines in the cloud. In particular, Chuck and I will work closely with our NASA partners who use Cumulus to manage the storage, processing, and distribution of their Earth science data.
…
Mark Boyd
Trying to solve more problems than I create
