Thanks @sbritt  for sharing the checklist above.

It looks like in order for the knowledge article from other KB's to surface on our custom portal , we need to include that knowledge base in the portal else it will continue to show the :knowledge record not found error. But this is making it to surface those article in the OOB search widget as well and restricting it exceptionally from there by using a custom search source is also not working.