It is true that finetuning would be faaaaaaar better than RAG, but it is just not feasible on a consumer device yet.
In my opinion RAG is definately not ideal but if you know how to use it then it can be quite useful. From my experience it is better to ask specific questions about the contents of any document, if the questions are too vague or generic it won't be able to find a suitable answer in its limited context window and will then begin to hallucinate.
In my opinion RAG is definately not ideal but if you know how to use it then it can be quite useful. From my experience it is better to ask specific questions about the contents of any document, if the questions are too vague or generic it won't be able to find a suitable answer in its limited context window and will then begin to hallucinate.