DynamoDB Performance with High-Fanout Replies

[Carol_7]

Hey everyone, I am having trouble with the performance of a DynamoDB table that stores posts and replies.

For posts, the PK is USER#<user-id> and the SK is POST#<post-id>#TIMESTAMP.

For replies, the PK is POST#<post-id> and the SK is REPLY#<reply-id>. #TIMESTAMP.

When there are thousands of replies to a post, it takes a long time to look up the most recent ones. Pagination only helps a little.

I'm not sure if it's better to put replies in a separate table or use a GSI.

Could anyone share tips, patterns, or experiences on how to handle high-fanout workloads in DynamoDB in the best way?

[rproffitt]

No code shared?

File under "Doctor, it hurts when I do this?"
Doctor: Don't do that.

[WendyDolan]

From my experience using fan-out via SQS or Lambda to pre-populate feeds works better than trying to query everything live. But it adds more moving parts.

[WendyDolan]

In my cass i noticed that batching writes and carefully designing partition keys helped a lot with performance. Still running into some hot partitions though curious how others tackle that.

[Contesthelper44]

If the number of replies grows significantly, I'd consider using a separate table. A GSI can improve some queries, but it's not a complete solution. Reviewing your partition key design is also important for better performance.

[Annie warner]

I would probably keep the replies in the same table unless there is a strong reason to separate them. The bigger issue sounds like the access pattern rather than the number of items itself.
For getting the latest replies, I would make sure the sort key is designed so DynamoDB can query them in descending order and use a small Limit rather than reading thousands of replies and then paginating through them.
A GSI can help if you have another access pattern for replies, but I would not add one just to solve a hot/high fanout partition. If a single post can receive a massive number of replies, you may also want to consider sharding the replies across multiple partition keys (for example, based on a reply bucket) and querying the relevant buckets in parallel.
So I would start with the query pattern and key design first then consider sharding or a separate table only if the traffic actually creates a hot partition or throughput issue.