this is not possible with unique fields. However, you could put all the stored vendor documents into a workflow. The workflow can then check if this specific combination of index fields already exists in the database.
If it does, the status could be changed to "duplicate" or something like this so you can easily search for this or display them in a task list to review.
Hope this helps.
Greetings from Germany,
Simon H. Hellmann
DocuWare System Consultant