ROW_NUMBER() and MAX per PARTITION?

This is my query:

select 
market,  
score, 
weeknum,
ROW_NUMBER() OVER (partition by weeknum ORDER BY weeknum, score asc) AS [OurRank]
from MyTable   

This query returns the following, but I need to calculate the column [NewCol] which is max([OurRank]) minus [OurRank] per week. In the following case, max([OurRank]) for week1 is 4 while max([OurRank]) for week2 is 6 since it's based on the number of markets :

Market      Score    Weeknum    OurRank  NewCol
Market1          2.40          1          1                 3  
Market3          2.50          1          2                 2  
Market4          2.70          1          3                 1  
Market2          2.86          1          4                 0  
Market3          2.30          2          1                 5  
Market5          2.40          2          2                 4  
Market2          2.50          2          3                 3  
Market4          2.66          2          4                 2  
Market1          2.76          2          5                 1  
Market6          2.86          2          6                 0  

Any help is aprpeciated.

VM