Hi @Peter Williams  
As Business rule condition is Insert and Update both hence when you update the approval record, script check for all attachments and update attachments in approval record which result in duplicate. If all attachments are added before the approval record creation you can update the business rule only on Insert condition so it will update attachment only once.

If my response proves useful, please indicate its helpfulness by selecting "Accept as Solution" and " Helpful."

Thanks!