Filter position

I have some derived table with many select statements inside:

SELECT Tk.productID, Tk.Q, Tk.col1  FROM
( 
  SELECT productID, Q, col1 from tbl1 where ....
  UNION ALL
  SELECT productID, Q, col1 from tbl2 where ....
  ....
)as Tk

It works fast. If I filter derived table:

SELECT Tk.productID, Tk.Q, Tk.col1 FROM
( 
  SELECT productID, Q, col1 from tbl1 where ....
  UNION ALL
  SELECT productID, Q, col1 from tbl2 where ....
  ....
)as Tk WHERE Tk.col1<>17

it works very slow. Is it possible to force optimizer to use col1 filter on derived table result? Otherwise it use this filter on every select inside derived table and since it is not indexed column, it takes a lot of time. The same works with CTE.

If i insert derived table into #tmp and than use filter there, it works fast:

SELECT Tk.productID, Tk.Q, Tk.col1 INTO #tmp FROM
( 
  SELECT productID, Q, col1 from tbl1 where ....
  UNION ALL
  SELECT productID, Q, col1 from tbl2 where ....
  ....
)as Tk 

SELECT * FROM #tmp WHERE col1<>17

So, is there some other way to force optimizer to use filter on the end result (on derived table) or i have to use temp table always in this kind of situations? I don't won't to put index on col1, since there is to many tables and indexes affected.