Exam AWS Certified AI Practitioner AIF-C01 topic 1 question 106 discussion

The ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score is widely used to measure the similarity between generated text and a set of reference texts. Since the company wants the LLM's output to resemble the provided readable examples, ROUGE is the most appropriate metric. ROUGE compares the LLM-generated text with the human-provided reference texts by evaluating n-gram overlap, precision, recall, and F1 score, making it a great choice for text coherence and readability assessment.

The correct answer is C. ROUGE score measures how well generated text matches reference examples.

Since the company wants the LLM output to resemble the provided examples in terms of coherence and readability, ROUGE score is the best metric for this evaluation.

he most suitable metric to assess whether the LLM output resembles the provided examples of more readable text is: C. Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score The ROUGE score is commonly used for evaluating the quality of text summarization and machine-generated text by comparing it to a set of reference texts. It measures how well the generated text matches the provided examples in terms of content and coherence. Specifically, ROUGE scores focus on the overlap of n-grams, word sequences, and word pairs between the generated text and the reference texts, making it ideal for this use case.