This phase I trial studies how well durvalumab with or without tremelimumab works in treating participants with stage II-IVA oropharyngeal squamous cell cancer. Immunotherapy with monoclonal antibodies, such as durvalumab and tremelimumab, may help the body's immune system attack the cancer, and may interfere with the ability of tumor cells to grow and spread.

PRIMARY OBJECTIVES: I. To assess the differences between CD8+ tumor infiltrating lymphocytes evaluated by immunohistochemistry staining in the post-treatment surgical specimens as compared to baseline in patients treated with durvalumab single agent compared with patients receiving durvalumab plus tremelimumab. SECONDARY OBJECTIVES: I. Safety and toxicity of durvalumab single agent or combined with tremelimumab administered every 4 weeks for 2 doses in the preoperative setting according to the National Cancer Institute Common Terminology Criteria for Adverse Events (CTCAE) version 4.03. II. Objective Response rate at 8 weeks, as determined by Response Evaluation Criteria in Solid Tumors (RECIST) 1.1. III. Percentage of patients undergoing the initially proposed surgery at 4 weeks. IV. Percentage of patients undergoing the initially proposed surgery at 8 weeks. V. Percentage of viable tumor cells in the surgical specimen. VI. Patient-reported outcomes (PRO) during treatment with checkpoint inhibitors. EXPLORATORY OBJECTIVES: I. To assess pre- and post- treatment tumor, blood and oral rinse based immune biomarkers. II. Correlate tissue and blood-based biomarkers with human papillomavirus (HPV) status, outcomes and toxicity. OUTLINE: Participants are randomized into 1 of 2 cohorts. COHORT I: Participants receive durvalumab intravenously (IV) over 1 hour on days 1 and 29 in the absence of disease progression or unaccepted toxicity. Between days 52 and 72, participants undergo standard of care surgery. COHORT II: Participants receive durvalumab IV over 1 hour and tremelimumab IV over 1 hour on days 1 and 29 in the absence of disease progression or unaccepted toxicity. Between days 52 and 72, participants undergo standard of care surgery. After completion of study treatment, participants are followed up at 28 and 42 days, 4 and 6 months, and then periodically for up to 5 years.