We only have one queue so not sure if this information will help.

In the queue settings, there is the schedule for live chat

In the Assignment rule, We have the timeout presence set to Available.

Even with these two settings, if an Agent is NOT available during the chat hours, the queue is closed, that is why we have the timeout setting to Available.

Since we only have one queue, I am sure there is more to this to meet your requirements. It should be in the Advanced Work Assignment areas.