Exam Professional Data Engineer topic 1 question 183 discussion

This is special case , plese Take a look carefully the below link and read at last paragraph at the bottom of this comment, let everyone share idea, We will go with B, C https://cloud.google.com/bigtable/docs/schema-design#time-based Don't use a timestamp by itself or at the beginning of a row key, because this will cause sequential writes to be pushed onto a single node, creating a hotspot. If you usually retrieve the most recent records first, you can use a reversed timestamp in the row key by subtracting the timestamp from your programming language's maximum value for long integers (in Java, java.lang.Long.MAX_VALUE). With a reversed timestamp, the records will be ordered from most recent to least recent.

B is the answer. https://cloud.google.com/bigtable/docs/schema-design#time-based If you usually retrieve the most recent records first, you can use a reversed timestamp in the row key by subtracting the timestamp from your programming language's maximum value for long integers (in Java, java.lang.Long.MAX_VALUE). With a reversed timestamp, the records will be ordered from most recent to least recent.

I don't think any answer is correct. A lot people upvote for B, but based on https://cloud.google.com/bigtable/docs/schema-design#time-based, "As with any timestamp, avoid starting a row key with a reversed timestamp so that you don't cause hotspots."

Why other options are not as suitable: A and B (One table for all indices): Storing all indices in a single table can lead to performance issues as the table grows larger. It also makes it harder to scale individual indices independently. C (Timestamp as row key): Using a regular timestamp would place the most recent data at the end of the table, making it less efficient to retrieve the latest prices.

Option B and Option D are both from my point of view correct. It depens on the situation. If there is need to get the information from each stock index, then D is more suitable. Otherwise B.

1. Reverse Timestamp for most recent stock prices 2. Having different table for each stock is more efficient, improves the query performance and option B doesn't specify stock in row key.

Row keys that start with a timestamp (irrespective reversed or not) causes sequential writes to be pushed onto a single node, creating a hotspot. If you put a timestamp in a row key, precede it with a high-cardinality value (index in our case) to avoid hotspots. The ideal option would be: "use the index and reversed timestamp as the row key design".

B is a correct answer because "you need to access only the most recent stock prices" "If you usually retrieve the most recent records first, you can use a reversed timestamp in the row key by subtracting the timestamp from your programming language's maximum value for long integers (in Java, java.lang.Long.MAX_VALUE). With a reversed timestamp, the records will be ordered from most recent to least recent." https://cloud.google.com/bigtable/docs/schema-design#time-based

B. One unique table for all indices, reverse timestamp as row key: A single table for all indices keeps the structure simple. Using a reverse timestamp as part of the row key ensures that the most recent data comes first in the sorted order. This design is beneficial for quickly accessing the latest data. For example: you can convert the timestamp to a string and format it in reverse order, like "yyyyMMddHHmmss", ensuring newer dates and times are sorted lexicographically before older ones.

Correct Is B

Option B using reverse timestamp only, this is not the answer. the right answer should be using the index and revers timestamp as the row key. So, Option D is the only answer, because not A,B,C .

https://cloud.google.com/bigtable/docs/schema-design#row-keys - If you usually retrieve the most recent records first, you can use a reversed timestamp B it is.

A is the answer

the answer relieves on whether the application need to access the whole indexes at the same time or not. If yes then is B, if no is A. in mind the answer is yes, so B makes more sense: I retrieve all the list at the same time.

https://cloud.google.com/bigtable/docs/schema-design#time-based If you usually retrieve the most recent records first, you can use a reversed timestamp in the row key by subtracting the timestamp from your programming language's maximum value for long integers (in Java, java.lang.Long.MAX_VALUE). With a reversed timestamp, the records will be ordered from most recent to least recent.

"access the data with the simplest query"

yes reverse time stamp is recommended to prevent hot spot. But our query pattern is we need most recent record the is easy when you use Timestamp and Also option a stating that our row key not starting with time stamp which is index#timestamp and which is the most efficient way for this scenario.