Use CASE in HAVING clause

I feel like this should be simple but it I'm just not grasping it. 

Say I have this query:

SELECT order_number, order_date FROM orders

I basically want to pull any order from the last 24 hours.  Well this is fairly easy for Tue-Fri but when I run the query on Mon, it will have zero results since there are no orders from the weekend.  If the query is ran on Mon it should query Fri instead.

I'm attempting to use case in my having clause but no luck. 

HAVING (DATEDIFF(dd, GETDATE(), order_date = (CASE order_date WHEN DATENAME(weekday,getdate()) = 'Friday' THEN '3' ELSE '0' END)