{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 0, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure the flour and milk are evenly combined with the mixture.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 1, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 6?", "why_question": "Why must Step 9 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 9 happen after Step 6?\n2. Why must Step 9 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the flour and milk are evenly combined with the mixture.\"\n}", "relevant_step_nos": [9, 6]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 2, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 6?", "why_question": "Why must Step 14 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 14 happen after Step 6?\n2. Why must Step 14 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 can happen after Step 12, as it is not dependent on Step 6.\"\n}", "relevant_step_nos": [14, 6]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 3, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure the flour and milk are evenly combined with the mixture.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 4, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 6?", "why_question": "Why must Step 11 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 11 happen after Step 6?\n2. Why must Step 11 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 6 because the mixture needs to be spooned into the tin after the ground almonds are stirred in.\"\n}", "relevant_step_nos": [11, 6]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 5, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 6?", "why_question": "Why must Step 13 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 13 happen after Step 6?\n2. Why must Step 13 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 does not need to happen after Step 6, as it can happen after Step 12 when the cake is fully baked.\"\n}", "relevant_step_nos": [13, 6]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 6, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the flour and milk are evenly combined with the other ingredients before adding the remaining flour and milk.\"\n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 7, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 8?", "why_question": "Why must Step 13 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 13 happen after Step 8?\n2. Why must Step 13 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 does not need to happen after Step 8, as it can happen after Step 12 when the cake is ready to be served.\"\n}", "relevant_step_nos": [13, 8]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 8, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 8?", "why_question": "Why must Step 14 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 14 happen after Step 8?\n2. Why must Step 14 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 can happen after Step 12, as it allows the cake to cool slightly before serving.\"\n}", "relevant_step_nos": [14, 8]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 9, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 8?", "why_question": "Why must Step 11 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 11 happen after Step 8?\n2. Why must Step 11 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure the apple slices are arranged over the cake mixture before baking.\"\n}", "relevant_step_nos": [11, 8]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 10, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 8?", "why_question": "Why must Step 12 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 12 happen after Step 8?\n2. Why must Step 12 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 8 because the mixture needs to be fully combined before baking.\"\n}", "relevant_step_nos": [12, 8]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 11, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 11?", "why_question": "Why must Step 12 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 12 happen after Step 11?\n2. Why must Step 12 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 11 because baking the cake is necessary before allowing it to cool and be served.\"\n}", "relevant_step_nos": [12, 11]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 12, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 5?", "why_question": "Why must Step 14 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 14 happen after Step 5?\n2. Why must Step 14 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 can happen after Step 12, as it allows the cake to cool slightly before serving.\"\n}", "relevant_step_nos": [14, 5]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 13, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 5?", "why_question": "Why must Step 11 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 11 happen after Step 5?\n2. Why must Step 11 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 5 because the mixture needs to be spooned into the tin after the eggs and ground almonds are added and combined.\"\n}", "relevant_step_nos": [11, 5]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 14, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because the ground almonds need to be stirred in after the eggs have been added and beaten well.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 15, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 5 to ensure that the flour and milk are added in the correct sequence and the batter is well combined.\"\n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 16, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the flour and milk are evenly combined with the other ingredients before adding the apples and baking the cake.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 17, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 4?", "why_question": "Why must Step 7 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 7 happen after Step 4?\n2. Why must Step 7 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 4 because the flour and milk need to be added gradually to the mixture to ensure it is well combined.\"\n}", "relevant_step_nos": [7, 4]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 18, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 4?", "why_question": "Why must Step 13 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 13 happen after Step 4?\n2. Why must Step 13 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 does not need to happen after Step 4, as it can happen after Step 12 when the cake is ready to be served.\"\n}", "relevant_step_nos": [13, 4]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 19, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 can happen after Step 4, but it does not have to happen immediately after Step 4.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 20, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 3?", "why_question": "Why must Step 10 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 10 happen after Step 3?\n2. Why must Step 10 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 3 because the cake tin needs to be prepared before arranging the apple in it.\"\n}", "relevant_step_nos": [10, 3]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 21, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 3?", "why_question": "Why must Step 12 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 12 happen after Step 3?\n2. Why must Step 12 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 3 because the cake needs to be baked after the batter is prepared and placed in the tin.\"\n}", "relevant_step_nos": [12, 3]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 22, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 9?", "why_question": "Why must Step 13 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 13 happen after Step 9?\n2. Why must Step 13 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 does not have to happen after Step 9, as it can happen after Step 12 when the cake is fully baked.\"\n}", "relevant_step_nos": [13, 9]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 23, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 7?", "why_question": "Why must Step 8 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 8 happen after Step 7?\n2. Why must Step 8 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the flour and milk are evenly combined with the mixture.\"\n}", "relevant_step_nos": [8, 7]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 24, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 7?", "why_question": "Why must Step 13 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 13 happen after Step 7?\n2. Why must Step 13 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 12 to allow the cake to cool slightly before turning onto a serving plate.\"\n}", "relevant_step_nos": [13, 7]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 25, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 13?", "why_question": "Why must Step 14 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 14 happen after Step 13?\n2. Why must Step 14 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen after Step 13 to allow the cake to cool slightly and set before serving.\"\n}", "relevant_step_nos": [14, 13]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 26, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 2?", "why_question": "Why must Step 14 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 14 happen after Step 2?\n2. Why must Step 14 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 must happen after Step 13, not Step 2, to allow the cake to cool slightly before serving.\"\n}", "relevant_step_nos": [14, 2]}
{"plan_idx": 1, "title": "almost no fat banana bread", "question_idx": 0, "steps": ["Preheat oven to 180 C / Gas mark 4.", "Lightly grease a 20x10cm (8x4 in) loaf tin.", "In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.", "Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.", "Pour batter into prepared tin.", "Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.", "Turn out onto wire rack and allow to cool before slicing."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almost no fat banana bread\n\nProcedure:\n1. Preheat oven to 180 C / Gas mark 4.\n2. Lightly grease a 20x10cm (8x4 in) loaf tin.\n3. In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.\n4. Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.\n5. Pour batter into prepared tin.\n6. Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.\n7. Turn out onto wire rack and allow to cool before slicing.\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 5 to allow the banana bread to cool and set before slicing.\"\n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 1, "title": "almost no fat banana bread", "question_idx": 1, "steps": ["Preheat oven to 180 C / Gas mark 4.", "Lightly grease a 20x10cm (8x4 in) loaf tin.", "In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.", "Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.", "Pour batter into prepared tin.", "Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.", "Turn out onto wire rack and allow to cool before slicing."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almost no fat banana bread\n\nProcedure:\n1. Preheat oven to 180 C / Gas mark 4.\n2. Lightly grease a 20x10cm (8x4 in) loaf tin.\n3. In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.\n4. Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.\n5. Pour batter into prepared tin.\n6. Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.\n7. Turn out onto wire rack and allow to cool before slicing.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the banana bread to cool and set before slicing.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 1, "title": "almost no fat banana bread", "question_idx": 2, "steps": ["Preheat oven to 180 C / Gas mark 4.", "Lightly grease a 20x10cm (8x4 in) loaf tin.", "In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.", "Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.", "Pour batter into prepared tin.", "Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.", "Turn out onto wire rack and allow to cool before slicing."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 2?", "why_question": "Why must Step 7 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almost no fat banana bread\n\nProcedure:\n1. Preheat oven to 180 C / Gas mark 4.\n2. Lightly grease a 20x10cm (8x4 in) loaf tin.\n3. In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.\n4. Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.\n5. Pour batter into prepared tin.\n6. Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.\n7. Turn out onto wire rack and allow to cool before slicing.\n\n\n1. Must Step 7 happen after Step 2?\n2. Why must Step 7 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 2 because the loaf needs to cool before slicing.\"\n}", "relevant_step_nos": [7, 2]}
{"plan_idx": 1, "title": "almost no fat banana bread", "question_idx": 3, "steps": ["Preheat oven to 180 C / Gas mark 4.", "Lightly grease a 20x10cm (8x4 in) loaf tin.", "In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.", "Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.", "Pour batter into prepared tin.", "Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.", "Turn out onto wire rack and allow to cool before slicing."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almost no fat banana bread\n\nProcedure:\n1. Preheat oven to 180 C / Gas mark 4.\n2. Lightly grease a 20x10cm (8x4 in) loaf tin.\n3. In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.\n4. Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.\n5. Pour batter into prepared tin.\n6. Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.\n7. Turn out onto wire rack and allow to cool before slicing.\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 3 because the batter needs to be prepared and poured into the tin before baking.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 1, "title": "almost no fat banana bread", "question_idx": 4, "steps": ["Preheat oven to 180 C / Gas mark 4.", "Lightly grease a 20x10cm (8x4 in) loaf tin.", "In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.", "Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.", "Pour batter into prepared tin.", "Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.", "Turn out onto wire rack and allow to cool before slicing."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almost no fat banana bread\n\nProcedure:\n1. Preheat oven to 180 C / Gas mark 4.\n2. Lightly grease a 20x10cm (8x4 in) loaf tin.\n3. In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.\n4. Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.\n5. Pour batter into prepared tin.\n6. Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.\n7. Turn out onto wire rack and allow to cool before slicing.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 3 because the ingredients mentioned in Step 4 need to be added to the mixture created in Step 3.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 2, "title": "apple-pumpkin-spread", "question_idx": 0, "steps": ["In a medium saucepan over medium heat, mix pumpkin puree, apple sauce, apple juice, dark brown soft sugar and spices.", "Bring mixture to the boil.", "Reduce heat and simmer approximately 1 1/2 hours.", "Transfer to sterile containers and chill in the refrigerator until serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: apple-pumpkin-spread\n\nProcedure:\n1. In a medium saucepan over medium heat, mix pumpkin puree, apple sauce, apple juice, dark brown soft sugar and spices.\n2. Bring mixture to the boil.\n3. Reduce heat and simmer approximately 1 1/2 hours.\n4. Transfer to sterile containers and chill in the refrigerator until serving.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 3 to allow the spread to cool and set in the refrigerator before serving.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 2, "title": "apple-pumpkin-spread", "question_idx": 1, "steps": ["In a medium saucepan over medium heat, mix pumpkin puree, apple sauce, apple juice, dark brown soft sugar and spices.", "Bring mixture to the boil.", "Reduce heat and simmer approximately 1 1/2 hours.", "Transfer to sterile containers and chill in the refrigerator until serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: apple-pumpkin-spread\n\nProcedure:\n1. In a medium saucepan over medium heat, mix pumpkin puree, apple sauce, apple juice, dark brown soft sugar and spices.\n2. Bring mixture to the boil.\n3. Reduce heat and simmer approximately 1 1/2 hours.\n4. Transfer to sterile containers and chill in the refrigerator until serving.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 to allow the mixture to cool and set before serving.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 0, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 6 because it involves adding the mozzarella, basil, and tomato on top of the cooked chicken.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 1, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 3 because the chicken needs to marinate in the vinaigrette for 12 to 24 hours before being saut\u00e9ed.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 2, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 3 because the chicken needs to be cooked before adding the mozzarella, basil, and tomato.\"\n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 3, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen after Step 1 because the vinaigrette needs to be poured over the chicken before it can be marinated in the fridge.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 4, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 1?", "why_question": "Why must Step 8 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 8 happen after Step 1?\n2. Why must Step 8 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 does not need to happen after Step 1, as it is the final step in the recipe and can be done after all the other steps are completed.\"\n}", "relevant_step_nos": [8, 1]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 5, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 1?", "why_question": "Why must Step 10 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 10 happen after Step 1?\n2. Why must Step 10 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 1 because the chicken needs to be marinated and cooked before it can be served.\"\n}", "relevant_step_nos": [10, 1]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 6, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because the chicken needs to be cooked before arranging it on the serving platter.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 7, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 5 because the mozzarella, basil, and tomato are added to the cooked chicken as a final step before serving.\"\n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 8, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 4 because the balsamic vinegar is drizzled over the chicken after it has been cooked and arranged on the serving platter.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 9, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 4?", "why_question": "Why must Step 7 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 7 happen after Step 4?\n2. Why must Step 7 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 4 because the chicken needs to be cooked before adding the mozzarella, basil, and tomato.\"\n}", "relevant_step_nos": [7, 4]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 10, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 8?", "why_question": "Why must Step 10 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 10 happen after Step 8?\n2. Why must Step 10 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 8 to ensure that the balsamic vinegar, salt, and pepper are added as the final seasoning before serving.\"\n}", "relevant_step_nos": [10, 8]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 11, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 2?", "why_question": "Why must Step 7 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 7 happen after Step 2?\n2. Why must Step 7 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 does not need to happen after Step 2, as marinating the chicken and adding the mozzarella can be done separately.\"\n}", "relevant_step_nos": [7, 2]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 12, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the chicken needs to marinate in the vinaigrette for 12 to 24 hours before being removed and cooked.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 13, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 9?", "why_question": "Why must Step 10 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 10 happen after Step 9?\n2. Why must Step 10 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 9 because the balsamic vinegar needs to be drizzled over the platter before seasoning with salt and pepper and serving.\"\n}", "relevant_step_nos": [10, 9]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 14, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 7?", "why_question": "Why must Step 10 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 10 happen after Step 7?\n2. Why must Step 10 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 7 because the balsamic vinegar is drizzled over the platter after the mozzarella, basil, and tomato are arranged on top of the chicken.\"\n}", "relevant_step_nos": [10, 7]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 0, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the base needs to be baked before adding the filling and toppings.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 1, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 2?", "why_question": "Why must Step 11 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 11 happen after Step 2?\n2. Why must Step 11 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 must happen after Step 8, not Step 2, because it involves covering the cake with whipped cream after all the other layers have been added.\"\n}", "relevant_step_nos": [11, 2]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 2, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 2?", "why_question": "Why must Step 9 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 9 happen after Step 2?\n2. Why must Step 9 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 must happen after Step 8, not Step 2.\"\n}", "relevant_step_nos": [9, 2]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 3, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 2 because the base needs to be baked before it can be cooled and used as a foundation for the rest of the cake.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 4, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 8 because the filling needs to be spread over the cooled base before arranging the bananas and pineapple on top.\"\n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 5, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 8?", "why_question": "Why must Step 11 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 11 happen after Step 8?\n2. Why must Step 11 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 8 because the whipped cream is the final topping for the banana split cake.\"\n}", "relevant_step_nos": [11, 8]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 6, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 8?", "why_question": "Why must Step 10 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 10 happen after Step 8?\n2. Why must Step 10 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 8 because the whipped cream needs to be the final layer on the cake.\"\n}", "relevant_step_nos": [10, 8]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 7, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 1?", "why_question": "Why must Step 4 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 4 happen after Step 1?\n2. Why must Step 4 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 1 because the oven needs to be preheated before baking the mixture.\"\n}", "relevant_step_nos": [4, 1]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 8, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 1?", "why_question": "Why must Step 8 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 8 happen after Step 1?\n2. Why must Step 8 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 must happen after Step 4, not Step 1.\"\n}", "relevant_step_nos": [8, 1]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 9, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 1?", "why_question": "Why must Step 11 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 11 happen after Step 1?\n2. Why must Step 11 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 must happen after Step 1 because it involves covering the cake with whipped cream, which is the final step in the procedure.\"\n}", "relevant_step_nos": [11, 1]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 10, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 4?", "why_question": "Why must Step 10 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 10 happen after Step 4?\n2. Why must Step 10 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 4 because the base needs to be cooled before adding the bananas and other fruits.\"\n}", "relevant_step_nos": [10, 4]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 11, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 4?", "why_question": "Why must Step 11 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 11 happen after Step 4?\n2. Why must Step 11 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 4 because the base needs to be cooled before adding the whipped cream.\"\n}", "relevant_step_nos": [11, 4]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 12, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 5?", "why_question": "Why must Step 10 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 10 happen after Step 5?\n2. Why must Step 10 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 must happen after Step 8, not Step 5.\"\n}", "relevant_step_nos": [10, 5]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 13, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 7?", "why_question": "Why must Step 9 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 9 happen after Step 7?\n2. Why must Step 9 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 7 because the filling needs to be spread over the cooled base before arranging the bananas and pineapple on top.\"\n}", "relevant_step_nos": [9, 7]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 14, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 6 because the ingredients need to be combined and beaten before spreading over the cooled base.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 5, "title": "best-ever-margarita", "question_idx": 0, "steps": ["Rim 2 margarita glasses with salt if desired and fill with ice.", "Pour the tequila, lime cordial, triple sec and lemon-lime soft drink into a cocktail shaker filled with ice.", "Hold your hand firmly over over the top of the shaker so the top doesn't pop off from the fizzy soft drink and shake vigorously.", "Pour into prepared margarita glasses and serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: best-ever-margarita\n\nProcedure:\n1. Rim 2 margarita glasses with salt if desired and fill with ice.\n2. Pour the tequila, lime cordial, triple sec and lemon-lime soft drink into a cocktail shaker filled with ice.\n3. Hold your hand firmly over over the top of the shaker so the top doesn't pop off from the fizzy soft drink and shake vigorously.\n4. Pour into prepared margarita glasses and serve.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the ingredients need to be mixed in the cocktail shaker before being poured into the prepared glasses.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 6, "title": "black-rice-pudding", "question_idx": 0, "steps": ["Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.", "Bring to the boil over high heat, stirring frequently.", "Reduce heat to medium-low.", "Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.", "Stir in coconut cream and vanilla extract.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: black-rice-pudding\n\nProcedure:\n1. Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.\n2. Bring to the boil over high heat, stirring frequently.\n3. Reduce heat to medium-low.\n4. Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.\n5. Stir in coconut cream and vanilla extract.\n6. Serve warm.\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"Yes\",\n  \"why_answer\": \"Step 2 must happen after Step 1 because the rice needs to be combined with sugar and water before bringing to the boil.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 6, "title": "black-rice-pudding", "question_idx": 1, "steps": ["Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.", "Bring to the boil over high heat, stirring frequently.", "Reduce heat to medium-low.", "Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.", "Stir in coconut cream and vanilla extract.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 1?", "why_question": "Why must Step 5 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: black-rice-pudding\n\nProcedure:\n1. Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.\n2. Bring to the boil over high heat, stirring frequently.\n3. Reduce heat to medium-low.\n4. Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.\n5. Stir in coconut cream and vanilla extract.\n6. Serve warm.\n\n\n1. Must Step 5 happen after Step 1?\n2. Why must Step 5 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 1 because the rice needs to be cooked and tender before adding the coconut cream and vanilla extract.\"\n}", "relevant_step_nos": [5, 1]}
{"plan_idx": 6, "title": "black-rice-pudding", "question_idx": 2, "steps": ["Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.", "Bring to the boil over high heat, stirring frequently.", "Reduce heat to medium-low.", "Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.", "Stir in coconut cream and vanilla extract.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: black-rice-pudding\n\nProcedure:\n1. Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.\n2. Bring to the boil over high heat, stirring frequently.\n3. Reduce heat to medium-low.\n4. Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.\n5. Stir in coconut cream and vanilla extract.\n6. Serve warm.\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 1 because the rice needs to be combined with sugar and water before it can be cooked to tenderness and creaminess.\"\n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 6, "title": "black-rice-pudding", "question_idx": 3, "steps": ["Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.", "Bring to the boil over high heat, stirring frequently.", "Reduce heat to medium-low.", "Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.", "Stir in coconut cream and vanilla extract.", "Serve warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: black-rice-pudding\n\nProcedure:\n1. Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.\n2. Bring to the boil over high heat, stirring frequently.\n3. Reduce heat to medium-low.\n4. Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.\n5. Stir in coconut cream and vanilla extract.\n6. Serve warm.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 5 must happen after Step 4 to ensure the rice is tender and creamy before adding the coconut cream and vanilla extract.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 0, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 4?", "why_question": "Why must Step 10 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 10 happen after Step 4?\n2. Why must Step 10 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 4 because the shortbread dough needs to be prepared and shaped before baking in the oven.\"\n}", "relevant_step_nos": [10, 4]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 1, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 4?", "why_question": "Why must Step 7 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 7 happen after Step 4?\n2. Why must Step 7 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 4 because the dough needs to be mixed well before rolling it out to the desired thickness.\"\n}", "relevant_step_nos": [7, 4]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 2, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 4 because the shortbread needs to be cut and prepared for baking before being placed on the baking trays.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 3, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 2 because the flour needs to be gradually incorporated into the butter and brown sugar mixture to form a soft dough.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 4, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 2?", "why_question": "Why must Step 9 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 9 happen after Step 2?\n2. Why must Step 9 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 must happen after Step 8, not Step 2, because it involves placing the shortbread on baking trays and baking them, which comes after the dough is prepared and cut into strips.\"\n}", "relevant_step_nos": [9, 2]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 5, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 8?", "why_question": "Why must Step 10 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 10 happen after Step 8?\n2. Why must Step 10 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 8 because the shortbread needs to be cut into strips before baking.\"\n}", "relevant_step_nos": [10, 8]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 6, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 6?", "why_question": "Why must Step 10 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 10 happen after Step 6?\n2. Why must Step 10 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 6 because the dough needs to be rolled and cut before baking.\"\n}", "relevant_step_nos": [10, 6]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 7, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 6 because the dough needs to be kneaded before it is rolled out to the desired thickness.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 8, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 6 because the dough needs to be rolled out to the specified thickness before cutting it into strips.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 9, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 3?", "why_question": "Why must Step 9 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 9 happen after Step 3?\n2. Why must Step 9 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 3 because the shortbread needs to be prepared and shaped before baking.\" \n}", "relevant_step_nos": [9, 3]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 10, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 3 because the flour needs to be added to the creamed butter and brown sugar mixture to form the dough.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 11, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 3?", "why_question": "Why must Step 8 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 8 happen after Step 3?\n2. Why must Step 8 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 3 because the dough needs to be rolled and cut into strips before baking.\"\n}", "relevant_step_nos": [8, 3]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 12, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 3 because the flour needs to be gradually added and kneaded to form the dough.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 13, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 3?", "why_question": "Why must Step 10 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 10 happen after Step 3?\n2. Why must Step 10 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 3 because the shortbread dough needs to be prepared and shaped before baking.\" \n}", "relevant_step_nos": [10, 3]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 14, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 5?", "why_question": "Why must Step 10 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 10 happen after Step 5?\n2. Why must Step 10 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 5 because the shortbread needs to be rolled out and cut before baking.\" \n}", "relevant_step_nos": [10, 5]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 15, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because the remaining flour is used for kneading the dough to the right consistency.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 16, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 9?", "why_question": "Why must Step 10 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 10 happen after Step 9?\n2. Why must Step 10 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 9 because the shortbread needs to be placed on the baking trays before it can be baked in the oven.\"\n}", "relevant_step_nos": [10, 9]}
{"plan_idx": 8, "title": "busters-liver-dinner", "question_idx": 0, "steps": ["Put all ingredients into a slow cooker, mix together.", "Put slow cooker on high for 4 to 5 hours or low all day.", "( If mixture starts to look too dry just add a little hot water.", ")", "Once cooked, divide between several plastic microwaveable tubs with lids.", "This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.", "When ready to eat microwave with lid half off for 2 to 3 minutes on medium high."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: busters-liver-dinner\n\nProcedure:\n1. Put all ingredients into a slow cooker, mix together.\n2. Put slow cooker on high for 4 to 5 hours or low all day.\n3. ( If mixture starts to look too dry just add a little hot water.\n4. )\n5. Once cooked, divide between several plastic microwaveable tubs with lids.\n6. This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.\n7. When ready to eat microwave with lid half off for 2 to 3 minutes on medium high.\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 3 can happen at any time during the cooking process if the mixture looks too dry.\"\n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 8, "title": "busters-liver-dinner", "question_idx": 1, "steps": ["Put all ingredients into a slow cooker, mix together.", "Put slow cooker on high for 4 to 5 hours or low all day.", "( If mixture starts to look too dry just add a little hot water.", ")", "Once cooked, divide between several plastic microwaveable tubs with lids.", "This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.", "When ready to eat microwave with lid half off for 2 to 3 minutes on medium high."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: busters-liver-dinner\n\nProcedure:\n1. Put all ingredients into a slow cooker, mix together.\n2. Put slow cooker on high for 4 to 5 hours or low all day.\n3. ( If mixture starts to look too dry just add a little hot water.\n4. )\n5. Once cooked, divide between several plastic microwaveable tubs with lids.\n6. This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.\n7. When ready to eat microwave with lid half off for 2 to 3 minutes on medium high.\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 5 because Step 5 involves dividing the cooked mixture into tubs, and Step 7 involves microwaving the meal in those tubs before eating.\"\n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 8, "title": "busters-liver-dinner", "question_idx": 2, "steps": ["Put all ingredients into a slow cooker, mix together.", "Put slow cooker on high for 4 to 5 hours or low all day.", "( If mixture starts to look too dry just add a little hot water.", ")", "Once cooked, divide between several plastic microwaveable tubs with lids.", "This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.", "When ready to eat microwave with lid half off for 2 to 3 minutes on medium high."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: busters-liver-dinner\n\nProcedure:\n1. Put all ingredients into a slow cooker, mix together.\n2. Put slow cooker on high for 4 to 5 hours or low all day.\n3. ( If mixture starts to look too dry just add a little hot water.\n4. )\n5. Once cooked, divide between several plastic microwaveable tubs with lids.\n6. This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.\n7. When ready to eat microwave with lid half off for 2 to 3 minutes on medium high.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 2 because it involves dividing the cooked mixture into portions for storage, which can only be done after the cooking process in Step 2 is completed.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 9, "title": "butternut-squash-pastry-for-pies", "question_idx": 0, "steps": ["Place the flour, egg and squash into the container of a food processor.", "Pulse until mixed into a doughy ball.", "You may need to add a little extra flour.", "Divide into two equal parts.", "Use in a pie or pastie recipe."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: butternut-squash-pastry-for-pies\n\nProcedure:\n1. Place the flour, egg and squash into the container of a food processor.\n2. Pulse until mixed into a doughy ball.\n3. You may need to add a little extra flour.\n4. Divide into two equal parts.\n5. Use in a pie or pastie recipe.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 5 can happen after Step 3, but it can also happen after Step 4 if the dough is divided before using in a recipe.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 9, "title": "butternut-squash-pastry-for-pies", "question_idx": 1, "steps": ["Place the flour, egg and squash into the container of a food processor.", "Pulse until mixed into a doughy ball.", "You may need to add a little extra flour.", "Divide into two equal parts.", "Use in a pie or pastie recipe."], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: butternut-squash-pastry-for-pies\n\nProcedure:\n1. Place the flour, egg and squash into the container of a food processor.\n2. Pulse until mixed into a doughy ball.\n3. You may need to add a little extra flour.\n4. Divide into two equal parts.\n5. Use in a pie or pastie recipe.\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen after Step 1 because the ingredients need to be mixed into a doughy ball before moving on to the next step.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 0, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 8?", "why_question": "Why must Step 16 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 16 happen after Step 8?\n2. Why must Step 16 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 16 must happen after Step 8 because Step 16 is the final step of baking the pie, while Step 8 is the preparation of the filling and sauce.\" \n}", "relevant_step_nos": [16, 8]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 1, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 8?", "why_question": "Why must Step 13 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 13 happen after Step 8?\n2. Why must Step 13 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 8 because the hot liquid mixture needs to be prepared before covering the pie with the top pastry.\"\n}", "relevant_step_nos": [13, 8]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 2, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 8?", "why_question": "Why must Step 15 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 15 happen after Step 8?\n2. Why must Step 15 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 8 because the filling needs to be thickened before baking the pie.\"\n}", "relevant_step_nos": [15, 8]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 3, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 9?", "why_question": "Why must Step 12 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 12 happen after Step 9?\n2. Why must Step 12 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 9 because the hot liquid mixture needs to be poured over the chicken mixture in the pie dish.\"\n}", "relevant_step_nos": [12, 9]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 4, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 9?", "why_question": "Why must Step 15 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 15 happen after Step 9?\n2. Why must Step 15 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 9 because the hot liquid mixture is poured over the chicken mixture in the pie dish before baking.\" \n}", "relevant_step_nos": [15, 9]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 5, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 9?", "why_question": "Why must Step 13 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 13 happen after Step 9?\n2. Why must Step 13 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 9 to ensure that the filling is properly sealed and the excess pastry is removed before baking.\"\n}", "relevant_step_nos": [13, 9]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 6, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 1?", "why_question": "Why must Step 15 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 15 happen after Step 1?\n2. Why must Step 15 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 1 because the oven needs to be preheated before baking the pie.\"\n}", "relevant_step_nos": [15, 1]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 7, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 2?", "why_question": "Why must Step 15 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 15 happen after Step 2?\n2. Why must Step 15 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 2 because the chicken and vegetable pie needs to be baked in the oven after the filling is prepared.\"\n}", "relevant_step_nos": [15, 2]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 8, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 2?", "why_question": "Why must Step 14 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 14 happen after Step 2?\n2. Why must Step 14 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 must happen after Step 10, not Step 2.\"\n}", "relevant_step_nos": [14, 2]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 9, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 4?", "why_question": "Why must Step 14 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 14 happen after Step 4?\n2. Why must Step 14 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen after Step 4 because the chicken mixture needs to be placed in the pie dish before making slits in the pastry and baking the pie.\"\n}", "relevant_step_nos": [14, 4]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 10, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 4?", "why_question": "Why must Step 16 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 16 happen after Step 4?\n2. Why must Step 16 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 4 because the pie needs to be baked before it can be cooled and served.\"\n}", "relevant_step_nos": [16, 4]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 11, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 4?", "why_question": "Why must Step 12 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 12 happen after Step 4?\n2. Why must Step 12 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 4 because the chicken mixture needs to be prepared and set aside before being placed in the pie dish.\"\n}", "relevant_step_nos": [12, 4]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 12, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 7?", "why_question": "Why must Step 14 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 14 happen after Step 7?\n2. Why must Step 14 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen after Step 7 because the hot liquid mixture needs to be poured over the chicken mixture before covering with the top pastry and baking.\"\n}", "relevant_step_nos": [14, 7]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 13, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 7?", "why_question": "Why must Step 9 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 9 happen after Step 7?\n2. Why must Step 9 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 7 because the liquid mixture needs to be thickened before being set aside.\"\n}", "relevant_step_nos": [9, 7]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 14, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 7?", "why_question": "Why must Step 8 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 8 happen after Step 7?\n2. Why must Step 8 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 requires the ingredients from Step 7 to be added slowly in order to thicken the mixture.\"\n}", "relevant_step_nos": [8, 7]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 15, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 7?", "why_question": "Why must Step 12 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 12 happen after Step 7?\n2. Why must Step 12 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 7 because the hot liquid mixture is prepared in Step 7 and needs to be poured over the chicken mixture in Step 12.\"\n}", "relevant_step_nos": [12, 7]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 16, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 5 because the liquid mixture needs to be prepared after cooking the onions in butter.\"\n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 17, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 5?", "why_question": "Why must Step 14 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 14 happen after Step 5?\n2. Why must Step 14 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen after Step 5 because Step 5 involves preparing the filling for the pie, and Step 14 involves baking the pie with the prepared filling.\" \n}", "relevant_step_nos": [14, 5]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 18, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 10?", "why_question": "Why must Step 11 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 11 happen after Step 10?\n2. Why must Step 11 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 10 because the prepared pie dish needs to be lined with pastry before placing the chicken mixture in it.\"\n}", "relevant_step_nos": [11, 10]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 19, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 10?", "why_question": "Why must Step 15 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 15 happen after Step 10?\n2. Why must Step 15 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 10 because the pie needs to be baked after the filling is added and covered with pastry.\"\n}", "relevant_step_nos": [15, 10]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 20, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 14?", "why_question": "Why must Step 16 happen after Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 16 happen after Step 14?\n2. Why must Step 16 happen after Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 14 to allow the pie to cool slightly before serving.\"\n}", "relevant_step_nos": [16, 14]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 21, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 12?", "why_question": "Why must Step 16 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 16 happen after Step 12?\n2. Why must Step 16 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 12 to allow the pie to cool slightly before serving.\"\n}", "relevant_step_nos": [16, 12]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 22, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 6?", "why_question": "Why must Step 16 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 16 happen after Step 6?\n2. Why must Step 16 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 16 must happen after Step 15, not Step 6, because Step 16 is the final step of the procedure after the pie has been baked.\"\n}", "relevant_step_nos": [16, 6]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 23, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 13?", "why_question": "Why must Step 14 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 14 happen after Step 13?\n2. Why must Step 14 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen after Step 13 to allow steam to escape during baking.\"\n}", "relevant_step_nos": [14, 13]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 24, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 13?", "why_question": "Why must Step 16 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 16 happen after Step 13?\n2. Why must Step 16 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 13 to allow the pie to cool slightly before serving.\"\n}", "relevant_step_nos": [16, 13]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 25, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 11?", "why_question": "Why must Step 13 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 13 happen after Step 11?\n2. Why must Step 13 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 11 to seal the edges and cut away excess pastry after placing the chicken mixture in the pie dish.\"\n}", "relevant_step_nos": [13, 11]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 26, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 3?", "why_question": "Why must Step 14 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 14 happen after Step 3?\n2. Why must Step 14 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 must happen after Step 10, not Step 3, because it involves baking the pie in the oven after assembling it.\"\n}", "relevant_step_nos": [14, 3]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 0, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 1?", "why_question": "Why must Step 8 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 8 happen after Step 1?\n2. Why must Step 8 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 must happen after Step 1 because it involves adding salt and sherry to the mixture in the pot, which is done after the initial cooking and draining of the prawns and chicken mixture.\"\n}", "relevant_step_nos": [8, 1]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 1, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The prawns and chicken mixture needs to be cooked before being added to the soup.\"\n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 2, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 1?", "why_question": "Why must Step 6 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 6 happen after Step 1?\n2. Why must Step 6 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 does not have to happen after Step 1, as it can happen after Step 12.\"\n}", "relevant_step_nos": [6, 1]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 3, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 1?", "why_question": "Why must Step 4 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 4 happen after Step 1?\n2. Why must Step 4 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 1 because the prawns and chicken mixture needs to be cooked before being added to the soup.\"\n}", "relevant_step_nos": [4, 1]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 4, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 1?", "why_question": "Why must Step 9 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 9 happen after Step 1?\n2. Why must Step 9 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 must happen after Step 1 because it involves simmering the soup, which is the final cooking step after all ingredients have been added.\"\n}", "relevant_step_nos": [9, 1]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 5, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 1?", "why_question": "Why must Step 7 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 7 happen after Step 1?\n2. Why must Step 7 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 does not have to happen after Step 1, as it can happen at any point during the cooking process.\"\n}", "relevant_step_nos": [7, 1]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 6, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 3?", "why_question": "Why must Step 12 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 12 happen after Step 3?\n2. Why must Step 12 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The rice needs to be added to the soup after the prawns and chicken mixture has been cooked and drained.\"\n}", "relevant_step_nos": [12, 3]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 7, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 does not have to happen after Step 3, as it can happen at any point before serving the soup.\"\n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 8, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 3?", "why_question": "Why must Step 8 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 8 happen after Step 3?\n2. Why must Step 8 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 does not have to happen after Step 3, as it can happen after Step 7 when the soup is brought to a boil again.\"\n}", "relevant_step_nos": [8, 3]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 9, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The prawns and chicken mixture needs to be cooked in the hot oil before draining and adding to the soup.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 10, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 3?", "why_question": "Why must Step 13 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 13 happen after Step 3?\n2. Why must Step 13 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 12, not Step 3, as it is the final step in the procedure.\"\n}", "relevant_step_nos": [13, 3]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 11, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 3 because the prawns and chicken mixture needs to be cooked before being added to the pot with the stock and other ingredients.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 12, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 must happen after Step 12 to ensure the rice is added to the soup before serving.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 13, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 2?", "why_question": "Why must Step 13 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 13 happen after Step 2?\n2. Why must Step 13 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 12, not Step 2.\"\n}", "relevant_step_nos": [13, 2]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 14, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 2?", "why_question": "Why must Step 8 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 8 happen after Step 2?\n2. Why must Step 8 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 must happen after Step 10, not Step 2.\"\n}", "relevant_step_nos": [8, 2]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 15, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the salt and sherry are added to the soup before it is brought to a boil and allowed to simmer.\"\n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 16, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 5?", "why_question": "Why must Step 12 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 12 happen after Step 5?\n2. Why must Step 12 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The rice needs to be added to the soup after the other ingredients have been cooked and the soup has been brought to a boil.\"\n}", "relevant_step_nos": [12, 5]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 17, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 4 to allow the flavors to develop and the ingredients to cook properly before adding the salt and sherry.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 18, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 7?", "why_question": "Why must Step 13 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 13 happen after Step 7?\n2. Why must Step 13 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 7 because the soup needs to be fully prepared before serving and enjoying.\"\n}", "relevant_step_nos": [13, 7]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 19, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 8?", "why_question": "Why must Step 13 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 13 happen after Step 8?\n2. Why must Step 13 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 8 because the soup needs to be fully cooked and ready to serve before enjoying.\"\n}", "relevant_step_nos": [13, 8]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 20, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 11?", "why_question": "Why must Step 13 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 13 happen after Step 11?\n2. Why must Step 13 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 11 because the rice needs to be browned quickly and then added to the soup before serving.\"\n}", "relevant_step_nos": [13, 11]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 21, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the salt and sherry are added to the boiling mixture before reducing the heat and allowing it to simmer.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 12, "title": "chicory-and-roquefort-salad", "question_idx": 0, "steps": ["In a bowl, toss chicory, Roquefort and pecan nuts together.", "Make the dressing by mixing together cream cheese, yoghurt and salt and pepper.", "Drizzle over salad."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicory-and-roquefort-salad\n\nProcedure:\n1. In a bowl, toss chicory, Roquefort and pecan nuts together.\n2. Make the dressing by mixing together cream cheese, yoghurt and salt and pepper.\n3. Drizzle over salad.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 2 because the dressing needs to be made before it can be drizzled over the salad.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 0, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 1?", "why_question": "Why must Step 21 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 21 happen after Step 1?\n2. Why must Step 21 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 21 must happen after Step 1 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [21, 1]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 1, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 1?", "why_question": "Why must Step 17 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 1?\n2. Why must Step 17 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 1 because the oven needs to be preheated before baking the cake.\"\n}", "relevant_step_nos": [17, 1]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 2, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 22 happen after Step 10?", "why_question": "Why must Step 22 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 22 happen after Step 10?\n2. Why must Step 22 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 22 must happen after Step 10 because it is the final step in the procedure and is not directly related to the mixing of the cake batter.\"\n}", "relevant_step_nos": [22, 10]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 3, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 10?", "why_question": "Why must Step 16 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 16 happen after Step 10?\n2. Why must Step 16 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 10 because the mixture needs to be poured into the cake tin after the cooled chocolate is added to the mix.\"\n}", "relevant_step_nos": [16, 10]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 4, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 10?", "why_question": "Why must Step 18 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 18 happen after Step 10?\n2. Why must Step 18 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 18 must happen after Step 10 because it involves baking the cake, which can only occur after the cake mixture has been prepared.\"\n}", "relevant_step_nos": [18, 10]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 5, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 10?", "why_question": "Why must Step 21 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 21 happen after Step 10?\n2. Why must Step 21 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [21, 10]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 6, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 10?", "why_question": "Why must Step 17 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 10?\n2. Why must Step 17 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 10 because the mixture needs to be poured into the cake tin before baking.\"\n}", "relevant_step_nos": [17, 10]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 7, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 12?", "why_question": "Why must Step 16 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 16 happen after Step 12?\n2. Why must Step 16 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 16 can happen after Step 12, but it does not have to happen immediately after Step 12.\"\n}", "relevant_step_nos": [16, 12]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 8, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 12?", "why_question": "Why must Step 17 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 12?\n2. Why must Step 17 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 12 because Step 12 involves beating egg whites until stiff, which is then folded into the cake mixture before baking.\"\n}", "relevant_step_nos": [17, 12]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 9, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 12?", "why_question": "Why must Step 21 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 21 happen after Step 12?\n2. Why must Step 21 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 21 must happen after Step 12 to allow the cake to set and cool completely before removing it from the tin.\"\n}", "relevant_step_nos": [21, 12]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 10, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 12?", "why_question": "Why must Step 18 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 18 happen after Step 12?\n2. Why must Step 18 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 18 must happen after Step 12 because Step 18 involves baking the cake, and Step 12 involves preparing the cake batter by folding in the egg white mixture.\" \n}", "relevant_step_nos": [18, 12]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 11, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 4?", "why_question": "Why must Step 18 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 18 happen after Step 4?\n2. Why must Step 18 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 18 must happen after Step 4 because the cake needs to be baked after the mixture is prepared and poured into the cake tin.\"\n}", "relevant_step_nos": [18, 4]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 12, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 22 happen after Step 4?", "why_question": "Why must Step 22 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 22 happen after Step 4?\n2. Why must Step 22 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 22 must happen after Step 4 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [22, 4]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 13, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 4?", "why_question": "Why must Step 17 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 4?\n2. Why must Step 17 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 4 because the cake mixture needs to be prepared and poured into the cake tin before baking.\"\n}", "relevant_step_nos": [17, 4]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 14, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 4?", "why_question": "Why must Step 13 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 13 happen after Step 4?\n2. Why must Step 13 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 requires the chocolate mixture to be prepared before folding in the egg whites.\"\n}", "relevant_step_nos": [13, 4]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 15, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 4?", "why_question": "Why must Step 21 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 21 happen after Step 4?\n2. Why must Step 21 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 21 must happen after Step 4 to allow the cake to set and cool completely before decorating.\"\n}", "relevant_step_nos": [21, 4]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 16, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 9?", "why_question": "Why must Step 17 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 9?\n2. Why must Step 17 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 9 because the cake mixture needs to be prepared and poured into the cake tin before baking.\"\n}", "relevant_step_nos": [17, 9]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 17, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 9?", "why_question": "Why must Step 16 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 16 happen after Step 9?\n2. Why must Step 16 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 9 because the mixture needs to be prepared before pouring it into the cake tin.\"\n}", "relevant_step_nos": [16, 9]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 18, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 9?", "why_question": "Why must Step 21 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 21 happen after Step 9?\n2. Why must Step 21 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cake to set before removing it from the tin.\"\n}", "relevant_step_nos": [21, 9]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 19, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 3?", "why_question": "Why must Step 21 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 21 happen after Step 3?\n2. Why must Step 21 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cake to set and cool completely before removing from the tin.\"\n}", "relevant_step_nos": [21, 3]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 20, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the chocolate mixture to cool before adding it to the other ingredients.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 21, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 3?", "why_question": "Why must Step 10 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 10 happen after Step 3?\n2. Why must Step 10 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 3 because the chocolate needs to be melted and then cooled before being added to the mixture.\"\n}", "relevant_step_nos": [10, 3]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 22, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 22 happen after Step 3?", "why_question": "Why must Step 22 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 22 happen after Step 3?\n2. Why must Step 22 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 22 must happen after Step 3 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [22, 3]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 23, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 requires the butter and margarine to be creamy, which is achieved by beating them together in Step 6.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 24, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 6?", "why_question": "Why must Step 13 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 13 happen after Step 6?\n2. Why must Step 13 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 requires the egg whites to be beaten separately, which is not mentioned in Step 6.\"\n}", "relevant_step_nos": [13, 6]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 25, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 6?", "why_question": "Why must Step 16 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 16 happen after Step 6?\n2. Why must Step 16 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 6 because the mixture needs to be prepared before pouring it into the cake tin.\"\n}", "relevant_step_nos": [16, 6]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 26, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 6?", "why_question": "Why must Step 18 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 18 happen after Step 6?\n2. Why must Step 18 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 18 must happen after Step 16, not Step 6.\"\n}", "relevant_step_nos": [18, 6]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 27, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 6?", "why_question": "Why must Step 17 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 6?\n2. Why must Step 17 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 6 because the cake mixture needs to be prepared and poured into the tin before baking.\"\n}", "relevant_step_nos": [17, 6]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 28, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 6?", "why_question": "Why must Step 21 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 21 happen after Step 6?\n2. Why must Step 21 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cake to set before removing it from the tin.\"\n}", "relevant_step_nos": [21, 6]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 29, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 6?", "why_question": "Why must Step 9 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 9 happen after Step 6?\n2. Why must Step 9 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 6 because the butter and margarine need to be beaten together before gradually adding the flour.\"\n}", "relevant_step_nos": [9, 6]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 30, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 6?", "why_question": "Why must Step 10 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 10 happen after Step 6?\n2. Why must Step 10 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 requires the cooled chocolate mixture, which is prepared in Step 2, so it must happen after Step 6.\"\n}", "relevant_step_nos": [10, 6]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 31, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 13?", "why_question": "Why must Step 17 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 13?\n2. Why must Step 17 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 13 because it involves baking the cake mixture in the oven, which is the final step after folding in the egg white mixture.\"\n}", "relevant_step_nos": [17, 13]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 32, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 22 happen after Step 13?", "why_question": "Why must Step 22 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 22 happen after Step 13?\n2. Why must Step 22 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 22 can happen after Step 13, but it does not have to happen immediately after Step 13.\"\n}", "relevant_step_nos": [22, 13]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 33, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 13?", "why_question": "Why must Step 21 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 21 happen after Step 13?\n2. Why must Step 21 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cake to set before removing it from the tin.\"\n}", "relevant_step_nos": [21, 13]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 34, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 22 happen after Step 5?", "why_question": "Why must Step 22 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 22 happen after Step 5?\n2. Why must Step 22 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 22 must happen after Step 5 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [22, 5]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 35, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 5?", "why_question": "Why must Step 13 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 13 happen after Step 5?\n2. Why must Step 13 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 requires the chocolate mixture to be cooled, which happens in Step 5.\"\n}", "relevant_step_nos": [13, 5]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 36, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 5?", "why_question": "Why must Step 18 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 18 happen after Step 5?\n2. Why must Step 18 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 18 must happen after Step 5 because the cake needs to be baked after the mixture is prepared and poured into the cake tin.\"\n}", "relevant_step_nos": [18, 5]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 37, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 5?", "why_question": "Why must Step 17 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 5?\n2. Why must Step 17 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The mixture needs to be poured into the cake tin before baking, which happens after it has been set aside to cool in Step 5.\"\n}", "relevant_step_nos": [17, 5]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 38, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 20 happen after Step 5?", "why_question": "Why must Step 20 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 20 happen after Step 5?\n2. Why must Step 20 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cake to set before removing it from the tin.\"\n}", "relevant_step_nos": [20, 5]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 39, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 7?", "why_question": "Why must Step 10 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 10 happen after Step 7?\n2. Why must Step 10 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 7 because the cooled chocolate needs to be gradually added to the butter, margarine, and sugar mixture to ensure it is well incorporated.\"\n}", "relevant_step_nos": [10, 7]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 40, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 7?", "why_question": "Why must Step 21 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 21 happen after Step 7?\n2. Why must Step 21 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The cake needs to cool completely before removing from the tin, so it must happen after Step 7.\"\n}", "relevant_step_nos": [21, 7]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 41, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 7?", "why_question": "Why must Step 17 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 7?\n2. Why must Step 17 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 7 because the cake mixture needs to be prepared before baking.\"\n}", "relevant_step_nos": [17, 7]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 42, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 7?", "why_question": "Why must Step 16 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 16 happen after Step 7?\n2. Why must Step 16 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 16 must happen after Step 7 because Step 16 involves pouring the mixture into the cake tin, while Step 7 involves beating in the sugar until light and fluffy.\"\n}", "relevant_step_nos": [16, 7]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 43, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 8?", "why_question": "Why must Step 10 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 10 happen after Step 8?\n2. Why must Step 10 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 8 because the cooled chocolate needs to be added to the mixture after the egg yolks and flour have been incorporated.\"\n}", "relevant_step_nos": [10, 8]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 44, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 22 happen after Step 8?", "why_question": "Why must Step 22 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 22 happen after Step 8?\n2. Why must Step 22 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 22 must happen after Step 8 because it is the final step in the procedure and is not directly related to Step 8.\"\n}", "relevant_step_nos": [22, 8]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 45, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 8?", "why_question": "Why must Step 16 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 16 happen after Step 8?\n2. Why must Step 16 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 16 must happen after Step 8 because Step 16 involves pouring the mixture into the cake tin, which can only be done after the mixture is prepared in Step 8.\"\n}", "relevant_step_nos": [16, 8]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 46, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 8?", "why_question": "Why must Step 17 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 8?\n2. Why must Step 17 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 8 because Step 8 involves adding the egg yolks and flour, which are essential ingredients for the cake batter, before baking in Step 17.\"\n}", "relevant_step_nos": [17, 8]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 47, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 8?", "why_question": "Why must Step 13 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 13 happen after Step 8?\n2. Why must Step 13 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 requires the egg whites to be beaten separately, which can only happen after the egg yolks have been added in Step 8.\"\n}", "relevant_step_nos": [13, 8]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 48, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 15?", "why_question": "Why must Step 17 happen after Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen after Step 15?\n2. Why must Step 17 happen after Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 15 because the mixture needs to be poured into the cake tin before baking.\"\n}", "relevant_step_nos": [17, 15]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 49, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 15?", "why_question": "Why must Step 21 happen after Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 21 happen after Step 15?\n2. Why must Step 21 happen after Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cake to set before removing it from the tin.\"\n}", "relevant_step_nos": [21, 15]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 50, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 2?", "why_question": "Why must Step 10 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 10 happen after Step 2?\n2. Why must Step 10 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Because the chocolate needs to be melted and cooled before being added to the mixture.\"\n}", "relevant_step_nos": [10, 2]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 51, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The chocolate mixture needs to cool before being added to the butter and sugar mixture.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 52, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 2?", "why_question": "Why must Step 13 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 13 happen after Step 2?\n2. Why must Step 13 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 2 because it involves folding egg white mixture into the chocolate mixture, which is a separate step from melting the chocolate chips, coffee, and water in Step 2.\"\n}", "relevant_step_nos": [13, 2]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 53, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 22 happen after Step 17?", "why_question": "Why must Step 22 happen after Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 22 happen after Step 17?\n2. Why must Step 22 happen after Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 22 must happen after Step 17 to allow the cake to cool completely before adding the icing sugar.\"\n}", "relevant_step_nos": [22, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 54, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 22 happen after Step 16?", "why_question": "Why must Step 22 happen after Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 22 happen after Step 16?\n2. Why must Step 22 happen after Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 22 must happen after Step 16 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [22, 16]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 55, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 11?", "why_question": "Why must Step 16 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 16 happen after Step 11?\n2. Why must Step 16 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 11 because the mixture needs to be poured into the cake tin after folding in the egg white mixture.\"\n}", "relevant_step_nos": [16, 11]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 56, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 22 happen after Step 18?", "why_question": "Why must Step 22 happen after Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 22 happen after Step 18?\n2. Why must Step 22 happen after Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 22 must happen after Step 18 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [22, 18]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 57, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_after", "binary_question": "Must Step 22 happen after Step 14?", "why_question": "Why must Step 22 happen after Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 22 happen after Step 14?\n2. Why must Step 22 happen after Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 22 must happen after Step 14 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [22, 14]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 0, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 10?", "why_question": "Why must Step 11 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 11 happen after Step 10?\n2. Why must Step 11 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To keep the pancakes warm while the rest are being cooked.\"\n}", "relevant_step_nos": [11, 10]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 1, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the ingredients to fully combine and for the gluten in the flour to relax, resulting in a smoother batter and better texture for the pancakes.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 2, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 3?", "why_question": "Why must Step 10 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 10 happen after Step 3?\n2. Why must Step 10 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 must happen after Step 9, not Step 3.\"\n}", "relevant_step_nos": [10, 3]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 3, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 3?", "why_question": "Why must Step 9 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 9 happen after Step 3?\n2. Why must Step 9 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 must happen after Step 7, not Step 3.\"\n}", "relevant_step_nos": [9, 3]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 4, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 3?", "why_question": "Why must Step 11 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 11 happen after Step 3?\n2. Why must Step 11 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 can happen after Step 3, but it is not a requirement for it to happen in that order.\"\n}", "relevant_step_nos": [11, 3]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 5, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 3?", "why_question": "Why must Step 8 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 8 happen after Step 3?\n2. Why must Step 8 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 must happen after Step 7, not Step 3.\"\n}", "relevant_step_nos": [8, 3]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 6, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 5?", "why_question": "Why must Step 9 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 9 happen after Step 5?\n2. Why must Step 9 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 5 because the frying pan needs to be heated before the pancake batter is poured into it.\"\n}", "relevant_step_nos": [9, 5]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 7, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because the frying pan needs to be heated before greasing it with sunflower oil.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 8, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 5 because the pan needs to be heated before pouring the batter.\"\n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 9, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 9?", "why_question": "Why must Step 11 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 11 happen after Step 9?\n2. Why must Step 11 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To keep the pancakes warm while the remaining batter is cooked.\"\n}", "relevant_step_nos": [11, 9]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 10, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 1?", "why_question": "Why must Step 11 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 11 happen after Step 1?\n2. Why must Step 11 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 can happen after Step 1, but it is not a requirement for the pancakes to be made.\" \n}", "relevant_step_nos": [11, 1]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 11, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 1?", "why_question": "Why must Step 4 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 4 happen after Step 1?\n2. Why must Step 4 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 4 can happen after Step 1, but it is not necessary for it to happen in that order.\"\n}", "relevant_step_nos": [4, 1]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 12, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 1?", "why_question": "Why must Step 10 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 10 happen after Step 1?\n2. Why must Step 10 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 1 because Step 10 involves using the batter created in Step 1 to make the pancakes.\"\n}", "relevant_step_nos": [10, 1]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 13, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 2?", "why_question": "Why must Step 10 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 10 happen after Step 2?\n2. Why must Step 10 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 must happen after Step 9, not Step 2.\"\n}", "relevant_step_nos": [10, 2]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 14, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 2?", "why_question": "Why must Step 9 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 9 happen after Step 2?\n2. Why must Step 9 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 must happen after Step 7, not Step 2.\"\n}", "relevant_step_nos": [9, 2]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 15, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 2?", "why_question": "Why must Step 7 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 7 happen after Step 2?\n2. Why must Step 7 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 2 because the batter needs to be prepared before pouring it into the pan.\"\n}", "relevant_step_nos": [7, 2]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 16, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 2 because the eggs need to be whisked in before adding the other ingredients.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 17, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the gluten in the flour to relax and the batter to thicken.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 18, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 2?", "why_question": "Why must Step 11 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 11 happen after Step 2?\n2. Why must Step 11 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 can happen after all the pancakes have been cooked, it does not have to happen after Step 2.\"\n}", "relevant_step_nos": [11, 2]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 19, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 7?", "why_question": "Why must Step 11 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 11 happen after Step 7?\n2. Why must Step 11 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To keep the pancakes warm while the rest are being cooked.\"\n}", "relevant_step_nos": [11, 7]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 20, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 must happen after Step 4 because the batter needs to rest for 20 minutes before cooking the pancakes.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 21, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 can happen after Step 4, but it doesn't have to happen immediately after Step 4.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 22, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 6?", "why_question": "Why must Step 9 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 9 happen after Step 6?\n2. Why must Step 9 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 6 because Step 6 prepares the pan for cooking the batter, and Step 9 involves flipping the pancake in the pan.\"\n}", "relevant_step_nos": [9, 6]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 23, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 8 because it is the process of flipping the pancake to cook the other side.\"\n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 15, "title": "cloud-bread", "question_idx": 0, "steps": ["Preheat oven to 150 C / Gas mark 2.", "Line a baking tray with baking paper.", "Mix cream cheese and egg yolks together until smooth.", "Place egg whites and baking powder in a glass, metal, or ceramic bowl;", "beat together until stiff peaks form.", "Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.", "Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.", "Bake in the preheated oven until golden brown, 17 to 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cloud-bread\n\nProcedure:\n1. Preheat oven to 150 C / Gas mark 2.\n2. Line a baking tray with baking paper.\n3. Mix cream cheese and egg yolks together until smooth.\n4. Place egg whites and baking powder in a glass, metal, or ceramic bowl;\n5. beat together until stiff peaks form.\n6. Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.\n7. Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.\n8. Bake in the preheated oven until golden brown, 17 to 20 minutes.\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 6 because the batter needs to be prepared and then baked in the oven.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 15, "title": "cloud-bread", "question_idx": 1, "steps": ["Preheat oven to 150 C / Gas mark 2.", "Line a baking tray with baking paper.", "Mix cream cheese and egg yolks together until smooth.", "Place egg whites and baking powder in a glass, metal, or ceramic bowl;", "beat together until stiff peaks form.", "Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.", "Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.", "Bake in the preheated oven until golden brown, 17 to 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cloud-bread\n\nProcedure:\n1. Preheat oven to 150 C / Gas mark 2.\n2. Line a baking tray with baking paper.\n3. Mix cream cheese and egg yolks together until smooth.\n4. Place egg whites and baking powder in a glass, metal, or ceramic bowl;\n5. beat together until stiff peaks form.\n6. Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.\n7. Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.\n8. Bake in the preheated oven until golden brown, 17 to 20 minutes.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 4 because the egg whites need to be beaten to stiff peaks before folding in the cream cheese and egg yolk mixture and baking the bread.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 15, "title": "cloud-bread", "question_idx": 2, "steps": ["Preheat oven to 150 C / Gas mark 2.", "Line a baking tray with baking paper.", "Mix cream cheese and egg yolks together until smooth.", "Place egg whites and baking powder in a glass, metal, or ceramic bowl;", "beat together until stiff peaks form.", "Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.", "Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.", "Bake in the preheated oven until golden brown, 17 to 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 4?", "why_question": "Why must Step 6 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cloud-bread\n\nProcedure:\n1. Preheat oven to 150 C / Gas mark 2.\n2. Line a baking tray with baking paper.\n3. Mix cream cheese and egg yolks together until smooth.\n4. Place egg whites and baking powder in a glass, metal, or ceramic bowl;\n5. beat together until stiff peaks form.\n6. Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.\n7. Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.\n8. Bake in the preheated oven until golden brown, 17 to 20 minutes.\n\n\n1. Must Step 6 happen after Step 4?\n2. Why must Step 6 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 4 because the egg whites need to be beaten to stiff peaks before being folded into the cream cheese and egg yolk mixture.\"\n}", "relevant_step_nos": [6, 4]}
{"plan_idx": 15, "title": "cloud-bread", "question_idx": 3, "steps": ["Preheat oven to 150 C / Gas mark 2.", "Line a baking tray with baking paper.", "Mix cream cheese and egg yolks together until smooth.", "Place egg whites and baking powder in a glass, metal, or ceramic bowl;", "beat together until stiff peaks form.", "Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.", "Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.", "Bake in the preheated oven until golden brown, 17 to 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cloud-bread\n\nProcedure:\n1. Preheat oven to 150 C / Gas mark 2.\n2. Line a baking tray with baking paper.\n3. Mix cream cheese and egg yolks together until smooth.\n4. Place egg whites and baking powder in a glass, metal, or ceramic bowl;\n5. beat together until stiff peaks form.\n6. Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.\n7. Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.\n8. Bake in the preheated oven until golden brown, 17 to 20 minutes.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 4 because the egg whites need to be beaten until stiff peaks form before folding them into the cream cheese and egg yolk mixture.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 15, "title": "cloud-bread", "question_idx": 4, "steps": ["Preheat oven to 150 C / Gas mark 2.", "Line a baking tray with baking paper.", "Mix cream cheese and egg yolks together until smooth.", "Place egg whites and baking powder in a glass, metal, or ceramic bowl;", "beat together until stiff peaks form.", "Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.", "Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.", "Bake in the preheated oven until golden brown, 17 to 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cloud-bread\n\nProcedure:\n1. Preheat oven to 150 C / Gas mark 2.\n2. Line a baking tray with baking paper.\n3. Mix cream cheese and egg yolks together until smooth.\n4. Place egg whites and baking powder in a glass, metal, or ceramic bowl;\n5. beat together until stiff peaks form.\n6. Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.\n7. Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.\n8. Bake in the preheated oven until golden brown, 17 to 20 minutes.\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 5 because the batter needs to be prepared and ready to be spooned onto the baking tray after the egg whites have been beaten to stiff peaks.\"\n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 0, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 6 because the whipped cream needs to be mixed with the other ingredients before spooning into serving bowls.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 1, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 6 because the whipped cream needs to be prepared before folding in the other ingredients.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 2, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 5?", "why_question": "Why must Step 10 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 10 happen after Step 5?\n2. Why must Step 10 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 does not need to happen after Step 5, as it is a separate garnishing step and can be done at any point after the dish is prepared.\"\n}", "relevant_step_nos": [10, 5]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 3, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 1?", "why_question": "Why must Step 4 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 4 happen after Step 1?\n2. Why must Step 4 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the toasted oats to cool before adding them to the whipped cream.\"\n}", "relevant_step_nos": [4, 1]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 4, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the oats to cool faster and prevent them from overcooking.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 5, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 2?", "why_question": "Why must Step 10 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 10 happen after Step 2?\n2. Why must Step 10 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 does not need to happen after Step 2, as it is a garnishing step and can be done at any point before serving.\"\n}", "relevant_step_nos": [10, 2]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 6, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 2?", "why_question": "Why must Step 7 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 7 happen after Step 2?\n2. Why must Step 7 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 requires the toasted oats from Step 3 to be folded into the whipped cream.\"\n}", "relevant_step_nos": [7, 2]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 7, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 2?", "why_question": "Why must Step 8 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 8 happen after Step 2?\n2. Why must Step 8 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 requires the toasted oats from Step 3 to be folded into the whipped cream.\"\n}", "relevant_step_nos": [8, 2]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 8, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Toasting the oats requires spreading them out in a thin layer on a baking tray first.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 9, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To cool the oats faster before adding them to the cream mixture.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 10, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The toasted oats need to cool before being folded into the whipped cream.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 11, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 3?", "why_question": "Why must Step 10 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 10 happen after Step 3?\n2. Why must Step 10 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 does not need to happen after Step 3, as it is a garnishing step and can be done after the dish is assembled.\"\n}", "relevant_step_nos": [10, 3]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 0, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 1?", "why_question": "Why must Step 5 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 5 happen after Step 1?\n2. Why must Step 5 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 1 because the stock needs to be dissolved in hot water before it can be added to the soup.\" \n}", "relevant_step_nos": [5, 1]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 1, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 1?", "why_question": "Why must Step 9 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 9 happen after Step 1?\n2. Why must Step 9 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 must happen after Step 8, not Step 1, because the soup needs to be pureed before being poured into bowls and garnished.\"\n}", "relevant_step_nos": [9, 1]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 2, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because the mixture needs to simmer for 20 minutes before being covered.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 3, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 5?", "why_question": "Why must Step 9 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 9 happen after Step 5?\n2. Why must Step 9 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 5 because the soup needs to be pureed before it is poured into bowls and garnished.\"\n}", "relevant_step_nos": [9, 5]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 4, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 5 because the mixture needs to simmer for 20 minutes after the stock and coriander are added before it can be pureed.\" \n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 5, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 4 because the soup needs to be pureed after the squash has been cooked and softened.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 6, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 4 because the stock needs to be added to the mixture of squash and other ingredients before covering and simmering.\" \n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 7, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 2?", "why_question": "Why must Step 8 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 8 happen after Step 2?\n2. Why must Step 8 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 must happen after Step 7, not Step 2, because the mixture needs to simmer for 20 minutes before being pureed.\"\n}", "relevant_step_nos": [8, 2]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 8, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 2 because the butter needs to be melted before cooking and stirring the onion, garlic, and crushed chillies.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 9, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 2?", "why_question": "Why must Step 7 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 7 happen after Step 2?\n2. Why must Step 7 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 2 because the squash needs to simmer in the stock for 20 minutes to fully cook and infuse the flavors.\"\n}", "relevant_step_nos": [7, 2]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 10, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 6?", "why_question": "Why must Step 9 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 9 happen after Step 6?\n2. Why must Step 9 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 6 because the soup needs to be simmered for 20 minutes before being pureed and served.\"\n}", "relevant_step_nos": [9, 6]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 11, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 3 because the squash needs to cook and soften before simmering in the stock.\"\n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 0, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 8 because it involves cooking the mussels until they have opened, which is the next step after wrapping the packets tightly.\" \n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 1, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 5?", "why_question": "Why must Step 10 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 10 happen after Step 5?\n2. Why must Step 10 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 5 to ensure that any mussels that do not open during cooking are discarded before serving.\" \n}", "relevant_step_nos": [10, 5]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 2, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because the curry mixture is dotted on the mussels after they have been divided into portions.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 3, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 is wrapping the mussels in foil, which must happen after adding the curry mixture in Step 5.\"\n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 4, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 2?", "why_question": "Why must Step 10 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 10 happen after Step 2?\n2. Why must Step 10 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 must happen after Step 9, not Step 2, to ensure that any unopened mussels are discarded before serving.\"\n}", "relevant_step_nos": [10, 2]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 5, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The curry mixture needs to be dotted on the mussels after they are divided into portions.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 6, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 7?", "why_question": "Why must Step 8 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 8 happen after Step 7?\n2. Why must Step 8 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 7 because wrapping the foil tightly around the portions is necessary to ensure the mussels cook properly and absorb the flavors of the curry mixture, red pepper, parsley, and lime slices.\"\n}", "relevant_step_nos": [8, 7]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 7, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 1?", "why_question": "Why must Step 10 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 10 happen after Step 1?\n2. Why must Step 10 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 must happen after Step 9, not Step 1, to ensure that any unopened mussels are discarded before serving.\" \n}", "relevant_step_nos": [10, 1]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 8, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 1?", "why_question": "Why must Step 9 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 9 happen after Step 1?\n2. Why must Step 9 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 1 because the barbecue needs to be preheated before cooking the mussels.\"\n}", "relevant_step_nos": [9, 1]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 9, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 6?", "why_question": "Why must Step 10 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 10 happen after Step 6?\n2. Why must Step 10 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 can happen after Step 9, as it is not directly dependent on Step 6.\"\n}", "relevant_step_nos": [10, 6]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 10, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 6?", "why_question": "Why must Step 9 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 9 happen after Step 6?\n2. Why must Step 9 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 6 because the mussels need to be cooked with the curry mixture and other ingredients before checking if they have opened.\"\n}", "relevant_step_nos": [9, 6]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 11, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 4 because it involves wrapping the mussels in foil, which is done after the mussels are divided into portions in Step 4.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 12, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 4?", "why_question": "Why must Step 10 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 10 happen after Step 4?\n2. Why must Step 10 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 4 because it is only after cooking the mussels that you can determine if any have not opened and need to be discarded.\"\n}", "relevant_step_nos": [10, 4]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 13, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 3?", "why_question": "Why must Step 10 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 10 happen after Step 3?\n2. Why must Step 10 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 does not need to happen after Step 3 because Step 10 is the final step after cooking the mussels, while Step 3 is the initial preparation of the foil.\" \n}", "relevant_step_nos": [10, 3]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 14, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 3?", "why_question": "Why must Step 8 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 8 happen after Step 3?\n2. Why must Step 8 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 3 because the mussels need to be prepared and seasoned before being wrapped in foil for cooking.\"\n}", "relevant_step_nos": [8, 3]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 15, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 3 because the mussels need to be prepared with the curry mixture before adding the red pepper, parsley, and lime slices.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 16, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 3 because the curry mixture needs to be prepared before it can be dotted on the mussels.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 17, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 3?", "why_question": "Why must Step 11 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 11 happen after Step 3?\n2. Why must Step 11 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 does not need to happen after Step 3, as it is the final step in the procedure and can be done after all other steps are completed.\"\n}", "relevant_step_nos": [11, 3]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 0, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 6 because the tart needs to be well set before serving topped with whipped cream.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 1, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 4 because the sliced strawberries need to be coated with the mixture before pouring into the base.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 2, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 4?", "why_question": "Why must Step 6 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 6 happen after Step 4?\n2. Why must Step 6 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 4 to allow the strawberry mixture to set and firm up in the fridge.\"\n}", "relevant_step_nos": [6, 4]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 3, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 2 because the mixture needs to be boiled before adding the strawberries and pouring into the base.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 4, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 2 because it involves removing the mixture from the heat after boiling for one minute.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 5, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 2 because the mixture needs to be chilled until well set after boiling for one minute.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 6, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 1 because the mixture needs to be heated and boiled before removing from heat and adding the strawberries.\"\n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 7, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen after Step 1 because the ingredients need to be mixed together before boiling.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 8, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 3 to allow the strawberry mixture to set properly before serving.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 9, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 involves heating the mixture, so it needs to cool down before adding the strawberries in Step 5.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 20, "title": "easy-coconut-rice", "question_idx": 0, "steps": ["Combine rice with the coconut milk, water, salt, sugar and cinnamon stick in a large saucepan with a tight-fitting lid.", "Bring to the boil; reduce the heat to a simmer and cover.", "Cook until the liquid is absorbed and the rice is tender, about 25 minutes.", "Remove cinnamon stick to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy-coconut-rice\n\nProcedure:\n1. Combine rice with the coconut milk, water, salt, sugar and cinnamon stick in a large saucepan with a tight-fitting lid.\n2. Bring to the boil; reduce the heat to a simmer and cover.\n3. Cook until the liquid is absorbed and the rice is tender, about 25 minutes.\n4. Remove cinnamon stick to serve.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the cinnamon stick needs to be removed before serving the coconut rice.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 0, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 6 because the almonds need to be sprinkled or rolled on the olive oil brushed spinach rolls before grilling.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 1, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 6 because grilling the spinach rolls is the final cooking step after brushing with olive oil.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 2, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 6?", "why_question": "Why must Step 9 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 9 happen after Step 6?\n2. Why must Step 9 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 6 to allow the spinach rolls to cool down before serving.\"\n}", "relevant_step_nos": [9, 6]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 3, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 5 because the almonds are sprinkled on top or rolled onto the spinach rolls after the cheese mix is wrapped in the spinach leaves.\"\n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 4, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 5?", "why_question": "Why must Step 9 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 9 happen after Step 5?\n2. Why must Step 9 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the spinach rolls to cool down before serving.\"\n}", "relevant_step_nos": [9, 5]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 5, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 2 to ensure that the cheese mix is thoroughly combined and creamy before proceeding to the next steps.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 6, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 2 because the cheese mix needs to be wrapped in the spinach leaves before brushing with olive oil.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 7, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 can happen after Step 3, but it does not have to happen immediately after Step 3.\"\n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 8, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the spinach rolls to cool down before serving.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 9, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 4 because the spinach leaves need to be blanched before wrapping the cheese mix in them.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 10, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 4 because the spinach rolls need to be blanched before grilling to ensure they are cooked properly.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 22, "title": "fiery-bean-and-avocado-dip", "question_idx": 0, "steps": ["In a large bowl, mix blackeyed beans, tomatoes, red onion, Scotch bonnet chillies, Italian dressing and cayenne pepper.", "Stir in the avocados just before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fiery-bean-and-avocado-dip\n\nProcedure:\n1. In a large bowl, mix blackeyed beans, tomatoes, red onion, Scotch bonnet chillies, Italian dressing and cayenne pepper.\n2. Stir in the avocados just before serving.\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The avocados should be stirred in just before serving to prevent them from browning and maintain their freshness.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 23, "title": "frugal-beef-stew", "question_idx": 0, "steps": ["Add the onion and beef to a frying pan with a little oil (if you feel it requires it, depends on how good your pan is) and cook until the onions are soft.", "Tip out the onion/beef into a casserole dish and add the rest of the ingredients to the frying pan with half the stock, cooking on a high heat for about 10 minutes.", "Pour the contents of the frying pan into the casserole dish along with the thyme and rest of the stock (and a little bit of pepper if you like) and put on the lid.", "Cook in the oven on a low heat (about 140 C) for around 3-4 hours, giving it a stir every hour."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: frugal-beef-stew\n\nProcedure:\n1. Add the onion and beef to a frying pan with a little oil (if you feel it requires it, depends on how good your pan is) and cook until the onions are soft.\n2. Tip out the onion/beef into a casserole dish and add the rest of the ingredients to the frying pan with half the stock, cooking on a high heat for about 10 minutes.\n3. Pour the contents of the frying pan into the casserole dish along with the thyme and rest of the stock (and a little bit of pepper if you like) and put on the lid.\n4. Cook in the oven on a low heat (about 140 C) for around 3-4 hours, giving it a stir every hour.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 3 because the stew needs to be cooked in the oven for 3-4 hours to become tender and flavorful.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 0, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 5?", "why_question": "Why must Step 17 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 17 happen after Step 5?\n2. Why must Step 17 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the tomato sauce.\"\n}", "relevant_step_nos": [17, 5]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 1, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 5?", "why_question": "Why must Step 18 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 18 happen after Step 5?\n2. Why must Step 18 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be added to the tomato sauce, which is prepared in Step 5, so Step 18 must happen after Step 5.\"\n}", "relevant_step_nos": [18, 5]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 2, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"The order of steps 5 and 8 does not matter as they are independent of each other.\"\n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 3, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The tomatoes need to be brought to a boil after being added to the saucepan.\"\n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 4, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 20 happen after Step 15?", "why_question": "Why must Step 20 happen after Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 20 happen after Step 15?\n2. Why must Step 20 happen after Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 20 must happen after Step 15 because the pasta needs to be cooked and tossed with the sauce before serving.\"\n}", "relevant_step_nos": [20, 15]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 5, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 15?", "why_question": "Why must Step 21 happen after Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 21 happen after Step 15?\n2. Why must Step 21 happen after Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 21 must happen after Step 15 because the meatballs and remaining sauce need to be spooned over the pasta after the meatballs have been cooked and drained.\"\n}", "relevant_step_nos": [21, 15]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 6, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 20 happen after Step 11?", "why_question": "Why must Step 20 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 20 happen after Step 11?\n2. Why must Step 20 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 20 can happen after Step 11, but it doesn't have to happen immediately after Step 11. The pasta can be cooked at any time before serving.\" \n}", "relevant_step_nos": [20, 11]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 7, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 11?", "why_question": "Why must Step 21 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 21 happen after Step 11?\n2. Why must Step 21 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 21 can happen after Step 11, but it does not have to happen immediately after Step 11.\"\n}", "relevant_step_nos": [21, 11]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 8, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 11?", "why_question": "Why must Step 18 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 18 happen after Step 11?\n2. Why must Step 18 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the tomato sauce in Step 18.\"\n}", "relevant_step_nos": [18, 11]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 9, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 16?", "why_question": "Why must Step 17 happen after Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 17 happen after Step 16?\n2. Why must Step 17 happen after Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be browned before being added to the tomato sauce to ensure they are fully cooked.\"\n}", "relevant_step_nos": [17, 16]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 10, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 16?", "why_question": "Why must Step 18 happen after Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 18 happen after Step 16?\n2. Why must Step 18 happen after Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be added to the tomato sauce and cooked for 20 minutes to ensure they are fully cooked and flavored by the sauce.\"\n}", "relevant_step_nos": [18, 16]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 11, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 3?", "why_question": "Why must Step 21 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 21 happen after Step 3?\n2. Why must Step 21 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 21 can happen after Step 19, as it is not directly dependent on Step 3.\"\n}", "relevant_step_nos": [21, 3]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 12, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 3?", "why_question": "Why must Step 15 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 15 happen after Step 3?\n2. Why must Step 15 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked after being shaped and before being added to the tomato sauce.\"\n}", "relevant_step_nos": [15, 3]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 13, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 20 happen after Step 3?", "why_question": "Why must Step 20 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 20 happen after Step 3?\n2. Why must Step 20 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 20 must happen after Step 19, not Step 3, because the pasta needs to be cooked and tossed with the sauce before serving.\" \n}", "relevant_step_nos": [20, 3]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 14, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 3?", "why_question": "Why must Step 18 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 18 happen after Step 3?\n2. Why must Step 18 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 18 must happen after Step 9, not Step 3.\"\n}", "relevant_step_nos": [18, 3]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 15, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 3?", "why_question": "Why must Step 13 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 13 happen after Step 3?\n2. Why must Step 13 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The onion mixture is needed to mix with the meatball ingredients in Step 12.\"\n}", "relevant_step_nos": [13, 3]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 16, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 14?", "why_question": "Why must Step 16 happen after Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 16 happen after Step 14?\n2. Why must Step 16 happen after Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be browned before being added to the tomato sauce.\"\n}", "relevant_step_nos": [16, 14]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 17, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 14?", "why_question": "Why must Step 21 happen after Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 21 happen after Step 14?\n2. Why must Step 21 happen after Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 21 must happen after Step 14 because the meatballs and remaining sauce need to be spooned over the pasta after the meatballs have been cooked and the sauce has been prepared.\"\n}", "relevant_step_nos": [21, 14]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 18, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 14?", "why_question": "Why must Step 17 happen after Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 17 happen after Step 14?\n2. Why must Step 17 happen after Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the tomato sauce.\"\n}", "relevant_step_nos": [17, 14]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 19, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 13?", "why_question": "Why must Step 21 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 21 happen after Step 13?\n2. Why must Step 21 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 21 must happen after Step 13 because the meatballs need to be cooked and added to the sauce before serving with the pasta.\"\n}", "relevant_step_nos": [21, 13]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 20, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 13?", "why_question": "Why must Step 15 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 15 happen after Step 13?\n2. Why must Step 15 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the tomato sauce.\"\n}", "relevant_step_nos": [15, 13]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 21, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 13?", "why_question": "Why must Step 17 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 17 happen after Step 13?\n2. Why must Step 17 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the tomato sauce.\"\n}", "relevant_step_nos": [17, 13]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 22, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 13?", "why_question": "Why must Step 18 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 18 happen after Step 13?\n2. Why must Step 18 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the tomato sauce.\"\n}", "relevant_step_nos": [18, 13]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 23, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 8?", "why_question": "Why must Step 10 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 10 happen after Step 8?\n2. Why must Step 10 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 can happen at any time during the 30-minute simmering period, it does not have to happen after Step 8.\"\n}", "relevant_step_nos": [10, 8]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 24, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 4?", "why_question": "Why must Step 13 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 13 happen after Step 4?\n2. Why must Step 13 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 requires the onion mixture from Step 3, which is created after Step 4.\"\n}", "relevant_step_nos": [13, 4]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 25, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 4?", "why_question": "Why must Step 16 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 16 happen after Step 4?\n2. Why must Step 16 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked after the onion mixture is set aside in the large bowl.\"\n}", "relevant_step_nos": [16, 4]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 26, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 20 happen after Step 6?", "why_question": "Why must Step 20 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 20 happen after Step 6?\n2. Why must Step 20 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 20 must happen after Step 6 because the pasta needs to be cooked and tossed with the sauce before serving.\"\n}", "relevant_step_nos": [20, 6]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 27, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 6?", "why_question": "Why must Step 9 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 9 happen after Step 6?\n2. Why must Step 9 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The sauce needs to be brought to a boil and then simmered before adding the basil and oregano.\"\n}", "relevant_step_nos": [9, 6]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 28, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 6?", "why_question": "Why must Step 17 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 17 happen after Step 6?\n2. Why must Step 17 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the tomato sauce.\"\n}", "relevant_step_nos": [17, 6]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 29, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 20 happen after Step 7?", "why_question": "Why must Step 20 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 20 happen after Step 7?\n2. Why must Step 20 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 20 must happen after Step 7 because the pasta needs to be cooked and tossed with the sauce before serving.\"\n}", "relevant_step_nos": [20, 7]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 30, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 7?", "why_question": "Why must Step 17 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 17 happen after Step 7?\n2. Why must Step 17 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be added to the tomato sauce after it has been simmered for 30 minutes.\"\n}", "relevant_step_nos": [17, 7]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 31, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 20 happen after Step 12?", "why_question": "Why must Step 20 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 20 happen after Step 12?\n2. Why must Step 20 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 20 can happen after Step 12, but it doesn't have to happen immediately after.\"\n}", "relevant_step_nos": [20, 12]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 32, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 12?", "why_question": "Why must Step 18 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 18 happen after Step 12?\n2. Why must Step 18 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the tomato sauce.\"\n}", "relevant_step_nos": [18, 12]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 33, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 21 happen after Step 2?", "why_question": "Why must Step 21 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 21 happen after Step 2?\n2. Why must Step 21 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 21 can happen after Step 2, but it does not have to happen immediately after Step 2.\"\n}", "relevant_step_nos": [21, 2]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 34, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 2?", "why_question": "Why must Step 12 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 12 happen after Step 2?\n2. Why must Step 12 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The onion mixture is needed for the meatballs, so it must be saut\u00e9ed before the meatball mixture is prepared.\"\n}", "relevant_step_nos": [12, 2]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 35, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because it involves transferring a portion of the onion mixture to a separate bowl, which can only be done after the onion mixture has been saut\u00e9ed in Step 2.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 36, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 1?", "why_question": "Why must Step 13 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 13 happen after Step 1?\n2. Why must Step 13 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 does not need to happen after Step 1 because they are independent steps in the procedure.\"\n}", "relevant_step_nos": [13, 1]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 37, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 1?", "why_question": "Why must Step 12 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 12 happen after Step 1?\n2. Why must Step 12 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 does not need to happen after Step 1 because they are separate parts of the recipe - one is for the sauce and the other is for the meatballs.\"\n}", "relevant_step_nos": [12, 1]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 38, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To transfer some of the onion mixture to a large bowl before adding the tomatoes to the saucepan.\"\n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 39, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 1?", "why_question": "Why must Step 4 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 4 happen after Step 1?\n2. Why must Step 4 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 4 does not need to happen after Step 1 because it is not dependent on the completion of Step 1.\"\n}", "relevant_step_nos": [4, 1]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 40, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To saut\u00e9 the onion and garlic in the large saucepan.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 41, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 18 happen after Step 10?", "why_question": "Why must Step 18 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 18 happen after Step 10?\n2. Why must Step 18 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be added to the tomato sauce to finish cooking and absorb the flavors.\"\n}", "relevant_step_nos": [18, 10]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 0, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to bring the water to a boil before adding the rice.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 1, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 5 because the tofu needs to be fried before seasoning with salt.\"\n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 2, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 5?", "why_question": "Why must Step 12 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 12 happen after Step 5?\n2. Why must Step 12 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 5 because the green curry sauce needs to be drizzled over the tofu and rice, which are prepared in Steps 5-11.\"\n}", "relevant_step_nos": [12, 5]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 3, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 11?", "why_question": "Why must Step 12 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 12 happen after Step 11?\n2. Why must Step 12 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 11 because the green curry paste needs to be mixed with the coconut milk and simmered before being drizzled over the tofu and rice to serve.\"\n}", "relevant_step_nos": [12, 11]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 4, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 8?", "why_question": "Why must Step 12 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 12 happen after Step 8?\n2. Why must Step 12 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 8 because the green curry sauce needs to be prepared before drizzling it over the tofu and rice.\"\n}", "relevant_step_nos": [12, 8]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 5, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 9?", "why_question": "Why must Step 10 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 10 happen after Step 9?\n2. Why must Step 10 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 9 because the green curry paste needs to be mixed into the boiling coconut milk to create the curry sauce.\"\n}", "relevant_step_nos": [10, 9]}
{"plan_idx": 26, "title": "halloumi-and-figs", "question_idx": 0, "steps": ["Preheat a grill for high heat.", "Break up the halloumi cheese, and toss with the fresh figs and walnuts in a small dish.", "Grill under the preheated grill until the halloumi is browned and the figs begin to caramelise, about 3 minutes.", "Drizzle with the honey."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: halloumi-and-figs\n\nProcedure:\n1. Preheat a grill for high heat.\n2. Break up the halloumi cheese, and toss with the fresh figs and walnuts in a small dish.\n3. Grill under the preheated grill until the halloumi is browned and the figs begin to caramelise, about 3 minutes.\n4. Drizzle with the honey.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the drizzling of honey is the final touch after grilling the halloumi, figs, and walnuts.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 0, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the dough needs to be formed before rolling in ground hazelnuts.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 1, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 2 because the dough needs to be chilled before forming the balls and making the hollows for the jam.\" \n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 2, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 2 because the biscuit dough needs to be formed into balls before making a hollow and filling with jam.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 3, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 1?", "why_question": "Why must Step 5 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 5 happen after Step 1?\n2. Why must Step 5 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 1 because the biscuit dough needs to be formed into balls before making a hollow and filling with jam.\"\n}", "relevant_step_nos": [5, 1]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 4, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Chilling the dough will make it easier to handle and prevent the biscuits from spreading too much during baking.\"\n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 5, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because the hollows for the jam are made in the balls of dough in Step 5, and then the jam is added in Step 6 before baking.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 29, "title": "lamb-curry--serves-4-", "question_idx": 0, "steps": ["Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.", "Fry the lamb in a little oil, then add the remaining onion.", "Gently fry for a minute, then add the paste and cook for a couple of minutes.", "Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.", "Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lamb-curry--serves-4-\n\nProcedure:\n1. Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.\n2. Fry the lamb in a little oil, then add the remaining onion.\n3. Gently fry for a minute, then add the paste and cook for a couple of minutes.\n4. Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.\n5. Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 adds the fresh tomatoes and spinach, which should be added towards the end of the cooking process to maintain their texture and flavor.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 29, "title": "lamb-curry--serves-4-", "question_idx": 1, "steps": ["Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.", "Fry the lamb in a little oil, then add the remaining onion.", "Gently fry for a minute, then add the paste and cook for a couple of minutes.", "Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.", "Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lamb-curry--serves-4-\n\nProcedure:\n1. Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.\n2. Fry the lamb in a little oil, then add the remaining onion.\n3. Gently fry for a minute, then add the paste and cook for a couple of minutes.\n4. Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.\n5. Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 3 because the paste needs to cook for a couple of minutes before adding the water and other ingredients.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 29, "title": "lamb-curry--serves-4-", "question_idx": 2, "steps": ["Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.", "Fry the lamb in a little oil, then add the remaining onion.", "Gently fry for a minute, then add the paste and cook for a couple of minutes.", "Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.", "Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lamb-curry--serves-4-\n\nProcedure:\n1. Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.\n2. Fry the lamb in a little oil, then add the remaining onion.\n3. Gently fry for a minute, then add the paste and cook for a couple of minutes.\n4. Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.\n5. Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the lamb needs to be fried and the paste needs to be cooked before adding the water, tomatoes, and other ingredients for simmering.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 0, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 4?", "why_question": "Why must Step 16 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 16 happen after Step 4?\n2. Why must Step 16 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 4 because the shortcakes need to be baked and cooled before assembling with the strawberries and whipped cream.\"\n}", "relevant_step_nos": [16, 4]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 1, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 involves kneading the dough, which can only be done after the dough is formed in Step 4.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 2, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 4 because Step 9 involves baking the dough that is formed in Step 4.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 3, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 3?", "why_question": "Why must Step 16 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 16 happen after Step 3?\n2. Why must Step 16 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 3 because the shortcakes need to be baked and cooled before assembling with the strawberries and whipped cream.\"\n}", "relevant_step_nos": [16, 3]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 4, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 involves dividing the dough, which is formed in Step 4, into halves and shaping them into circles.\"\n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 5, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 requires the flour mixture from Step 3 to be combined with the milk, egg yolk, and lemon zest.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 6, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 3?", "why_question": "Why must Step 10 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 10 happen after Step 3?\n2. Why must Step 10 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 must happen after Step 9 to allow the shortcakes to cool before assembling.\"\n}", "relevant_step_nos": [10, 3]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 7, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 12?", "why_question": "Why must Step 16 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 16 happen after Step 12?\n2. Why must Step 16 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 12 because the whipped cream needs to be prepared before assembling the shortcakes.\"\n}", "relevant_step_nos": [16, 12]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 8, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 7?", "why_question": "Why must Step 10 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 10 happen after Step 7?\n2. Why must Step 10 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The shortcakes need to cool for 15 minutes before assembling.\"\n}", "relevant_step_nos": [10, 7]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 9, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 7?", "why_question": "Why must Step 15 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 15 happen after Step 7?\n2. Why must Step 15 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 7 because the shortcakes need to be baked and cooled before assembling with the lemon butter, strawberries, and whipped cream.\"\n}", "relevant_step_nos": [15, 7]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 10, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 divides the dough into two portions before patting or rolling each half into a circle.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 11, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 5?", "why_question": "Why must Step 9 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 9 happen after Step 5?\n2. Why must Step 9 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 5 because the dough needs to be baked before assembling the shortcakes.\"\n}", "relevant_step_nos": [9, 5]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 12, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 requires the dough to be divided in half, which can only happen after the dough has been kneaded 10 times in Step 5.\"\n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 13, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 5?", "why_question": "Why must Step 15 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 15 happen after Step 5?\n2. Why must Step 15 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 5 because the shortcakes need to be baked and cooled before assembling with lemon butter, strawberries, and whipped cream.\"\n}", "relevant_step_nos": [15, 5]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 14, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 2?", "why_question": "Why must Step 14 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 14 happen after Step 2?\n2. Why must Step 14 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 must happen after Step 9, not Step 2.\"\n}", "relevant_step_nos": [14, 2]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 15, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 2?", "why_question": "Why must Step 8 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 8 happen after Step 2?\n2. Why must Step 8 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 2 because the dough needs to be formed and shaped before baking.\"\n}", "relevant_step_nos": [8, 2]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 16, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 requires the dry ingredients from Step 2 to be combined with the butter.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 17, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 2?", "why_question": "Why must Step 10 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 10 happen after Step 2?\n2. Why must Step 10 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 must happen after Step 9, not Step 2.\"\n}", "relevant_step_nos": [10, 2]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 18, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 2?", "why_question": "Why must Step 9 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 9 happen after Step 2?\n2. Why must Step 9 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 2 because the dough needs to be baked after it has been prepared.\"\n}", "relevant_step_nos": [9, 2]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 19, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 2?", "why_question": "Why must Step 16 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 16 happen after Step 2?\n2. Why must Step 16 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 2 because the shortcakes need to be baked and cooled before assembling with the strawberries and whipped cream.\"\n}", "relevant_step_nos": [16, 2]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 20, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 2 because the dough needs to be divided after it has been formed and kneaded.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 21, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 6?", "why_question": "Why must Step 16 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 16 happen after Step 6?\n2. Why must Step 16 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 6 because the shortcakes need to be split in half before assembling them with the strawberries and whipped cream.\"\n}", "relevant_step_nos": [16, 6]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 22, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 6?", "why_question": "Why must Step 15 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 15 happen after Step 6?\n2. Why must Step 15 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 6 because the dough needs to be divided and baked before assembling the shortcakes.\"\n}", "relevant_step_nos": [15, 6]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 23, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 requires the dough to be divided in half, which can only happen after the dough is formed in Step 6.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 24, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 13?", "why_question": "Why must Step 16 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 16 happen after Step 13?\n2. Why must Step 16 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 requires the whipped cream from Step 13 to be used in assembling the shortcakes.\"\n}", "relevant_step_nos": [16, 13]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 25, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 13?", "why_question": "Why must Step 15 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 15 happen after Step 13?\n2. Why must Step 15 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 13 because the whipped cream is prepared in Step 13 and is used in Step 15 to assemble the shortcakes.\"\n}", "relevant_step_nos": [15, 13]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 26, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 9?", "why_question": "Why must Step 15 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 15 happen after Step 9?\n2. Why must Step 15 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 9 because the shortcakes need to be baked and cooled before assembling them with the strawberries and whipped cream.\"\n}", "relevant_step_nos": [15, 9]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 27, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 9?", "why_question": "Why must Step 10 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 10 happen after Step 9?\n2. Why must Step 10 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the shortcakes to cool before assembling the dessert.\"\n}", "relevant_step_nos": [10, 9]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 28, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 9?", "why_question": "Why must Step 16 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 16 happen after Step 9?\n2. Why must Step 16 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 9 because the shortcakes need to cool before they can be assembled with the strawberries and whipped cream.\"\n}", "relevant_step_nos": [16, 9]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 29, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 14?", "why_question": "Why must Step 15 happen after Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 15 happen after Step 14?\n2. Why must Step 15 happen after Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 14 because it involves assembling the shortcakes with the lemon butter, strawberries, and whipped cream.\"\n}", "relevant_step_nos": [15, 14]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 30, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 14?", "why_question": "Why must Step 16 happen after Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 16 happen after Step 14?\n2. Why must Step 16 happen after Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 14 because it involves assembling the shortcakes with the strawberries and whipped cream.\"\n}", "relevant_step_nos": [16, 14]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 31, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 1?", "why_question": "Why must Step 16 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 16 happen after Step 1?\n2. Why must Step 16 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 16 must happen after Step 1 because the strawberries need to be refrigerated until serving, so assembling the shortcakes should happen after the strawberries have been prepared and refrigerated.\"\n}", "relevant_step_nos": [16, 1]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 32, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 8?", "why_question": "Why must Step 14 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 14 happen after Step 8?\n2. Why must Step 14 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen after Step 8 because the shortcakes need to be baked before they can be split in half and assembled.\"\n}", "relevant_step_nos": [14, 8]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 33, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 8?", "why_question": "Why must Step 16 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 16 happen after Step 8?\n2. Why must Step 16 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 8 because the shortcakes need to be baked and cooled before assembling the dessert.\"\n}", "relevant_step_nos": [16, 8]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 0, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 1?", "why_question": "Why must Step 11 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 11 happen after Step 1?\n2. Why must Step 11 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 must happen after Step 10 to ensure the soup is seasoned and reheated before serving.\"\n}", "relevant_step_nos": [11, 1]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 1, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 1?", "why_question": "Why must Step 9 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 9 happen after Step 1?\n2. Why must Step 9 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 must happen after Step 1 because it involves blending the soup, which can be done after the lentils and vegetables have cooked for 25-30 minutes.\"\n}", "relevant_step_nos": [9, 1]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 2, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To remove the bay leaf from the soup before blending it.\"\n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 3, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 5?", "why_question": "Why must Step 9 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 9 happen after Step 5?\n2. Why must Step 9 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 5 because the lentils and vegetables need to be cooked before blending the soup.\"\n}", "relevant_step_nos": [9, 5]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 4, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 6?", "why_question": "Why must Step 9 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 9 happen after Step 6?\n2. Why must Step 9 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 6 because it involves blending the soup after it has simmered and the lentils and vegetables are very soft.\"\n}", "relevant_step_nos": [9, 6]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 5, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 6?", "why_question": "Why must Step 11 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 11 happen after Step 6?\n2. Why must Step 11 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 6 to ensure that the soup is seasoned and reheated before serving.\"\n}", "relevant_step_nos": [11, 6]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 6, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 6?", "why_question": "Why must Step 12 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 12 happen after Step 6?\n2. Why must Step 12 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 6 because the soup needs to be reheated before serving.\"\n}", "relevant_step_nos": [12, 6]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 7, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 2 because the lentils, stock, tomatoes, and other ingredients need to be added after the vegetables have softened and the flavors have developed.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 8, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 2?", "why_question": "Why must Step 11 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 11 happen after Step 2?\n2. Why must Step 11 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 must happen after Step 9, not Step 2.\"\n}", "relevant_step_nos": [11, 2]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 9, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 2?", "why_question": "Why must Step 12 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 12 happen after Step 2?\n2. Why must Step 12 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 must happen after Step 9, not Step 2, because the soup needs to be blended and seasoned before serving.\"\n}", "relevant_step_nos": [12, 2]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 10, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 11?", "why_question": "Why must Step 12 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 12 happen after Step 11?\n2. Why must Step 12 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 11 to ensure that the soup is served hot and bubbling.\"\n}", "relevant_step_nos": [12, 11]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 11, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 4?", "why_question": "Why must Step 12 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 12 happen after Step 4?\n2. Why must Step 12 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 can happen after Step 9, as it is the final step in the procedure.\"\n}", "relevant_step_nos": [12, 4]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 12, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 must happen after Step 6 to ensure the lentils and vegetables are very soft before blending.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 13, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 4?", "why_question": "Why must Step 11 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 11 happen after Step 4?\n2. Why must Step 11 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 4 to ensure that the soup is seasoned to taste and reheated before serving.\"\n}", "relevant_step_nos": [11, 4]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 14, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 9?", "why_question": "Why must Step 11 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 11 happen after Step 9?\n2. Why must Step 11 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 9 to ensure that the soup is seasoned and reheated before serving.\"\n}", "relevant_step_nos": [11, 9]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 15, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 9?", "why_question": "Why must Step 12 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 12 happen after Step 9?\n2. Why must Step 12 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 9 because the soup needs to be blended before it can be served drizzled with coriander yoghurt and sprinkled with black pepper.\"\n}", "relevant_step_nos": [12, 9]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 16, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 9?", "why_question": "Why must Step 10 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 10 happen after Step 9?\n2. Why must Step 10 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 9 to check the consistency of the soup before diluting it with more stock.\"\n}", "relevant_step_nos": [10, 9]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 17, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 3 because the lentils and vegetables need to cook for 25-30 minutes, so they should be added after the initial vegetables have softened.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 18, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 3?", "why_question": "Why must Step 12 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 12 happen after Step 3?\n2. Why must Step 12 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 does not need to happen after Step 3, as it is the final step in the procedure and can be done after all other steps are completed.\"\n}", "relevant_step_nos": [12, 3]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 19, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Blending the soup is necessary before checking the consistency and adjusting it with more stock.\"\n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 20, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 8?", "why_question": "Why must Step 10 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 10 happen after Step 8?\n2. Why must Step 10 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 happens after Step 8 to ensure the soup reaches the desired consistency before seasoning and reheating.\"\n}", "relevant_step_nos": [10, 8]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 0, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 4?", "why_question": "Why must Step 7 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 7 happen after Step 4?\n2. Why must Step 7 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 4 because the dough needs to be formed before placing it on the baking tray.\"\n}", "relevant_step_nos": [7, 4]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 1, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 4 because the dough needs to be baked after it has been prepared.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 2, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 7?", "why_question": "Why must Step 9 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 9 happen after Step 7?\n2. Why must Step 9 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 7 because the rock cakes need to cool down after baking before they can be handled or eaten.\"\n}", "relevant_step_nos": [9, 7]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 3, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 7?", "why_question": "Why must Step 8 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 8 happen after Step 7?\n2. Why must Step 8 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 7 because the dough needs to be placed on the baking tray before baking for 20 minutes.\"\n}", "relevant_step_nos": [8, 7]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 4, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the dough balls have enough space to expand while baking and do not stick together.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 5, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 6 because the dough needs to be baked before it can be removed and placed on a cooling tray.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 6, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 3 because the dough needs to be formed before placing it on the baking tray.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 7, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 3 because the egg and currants need to be added to the flour and Splenda&reg; mixture to form the dough.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 8, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because the dough needs to be formed before placing it on the baking tray.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 9, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 8 to allow the rock cakes to cool down and set before handling them.\"\n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 10, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 1?", "why_question": "Why must Step 8 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 8 happen after Step 1?\n2. Why must Step 8 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 1 because the oven needs to be preheated before baking the rock cakes.\"\n}", "relevant_step_nos": [8, 1]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 11, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 2?", "why_question": "Why must Step 7 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 7 happen after Step 2?\n2. Why must Step 7 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 2 because the baking tray needs to be lined with baking parchment before placing the dough on it.\"\n}", "relevant_step_nos": [7, 2]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 12, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 2?", "why_question": "Why must Step 9 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 9 happen after Step 2?\n2. Why must Step 9 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 2 because the baking tray needs to be lined with baking parchment before placing the dough on it.\"\n}", "relevant_step_nos": [9, 2]}
{"plan_idx": 33, "title": "old-fashioned-welsh-rarebit", "question_idx": 0, "steps": ["Grate 250g well-flavoured mature Cheddar cheese (you can also use a good Double Gloucester or Shropshire) and put into a heavy-based medium-sized pan (preferably nonstick) with 10g butter, 1 tsp mustard (English, wholegrain or Dijon), plenty of ground black pepper and 4 tbsp ale or semi-skimmed milk.", "Set over a low heat and melt gently, stirring, until smooth and thick.", "Spoon onto 4 thick slices of toast and grill until browned and bubbling.", "Serve with grilled mushrooms and a leaf salad for a light lunch or snack."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: old-fashioned-welsh-rarebit\n\nProcedure:\n1. Grate 250g well-flavoured mature Cheddar cheese (you can also use a good Double Gloucester or Shropshire) and put into a heavy-based medium-sized pan (preferably nonstick) with 10g butter, 1 tsp mustard (English, wholegrain or Dijon), plenty of ground black pepper and 4 tbsp ale or semi-skimmed milk.\n2. Set over a low heat and melt gently, stirring, until smooth and thick.\n3. Spoon onto 4 thick slices of toast and grill until browned and bubbling.\n4. Serve with grilled mushrooms and a leaf salad for a light lunch or snack.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"Yes\",\n  \"why_answer\": \"Step 3 must happen after Step 2 because the mixture needs to be thickened and smooth before spooning onto the toast and grilling.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 33, "title": "old-fashioned-welsh-rarebit", "question_idx": 1, "steps": ["Grate 250g well-flavoured mature Cheddar cheese (you can also use a good Double Gloucester or Shropshire) and put into a heavy-based medium-sized pan (preferably nonstick) with 10g butter, 1 tsp mustard (English, wholegrain or Dijon), plenty of ground black pepper and 4 tbsp ale or semi-skimmed milk.", "Set over a low heat and melt gently, stirring, until smooth and thick.", "Spoon onto 4 thick slices of toast and grill until browned and bubbling.", "Serve with grilled mushrooms and a leaf salad for a light lunch or snack."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: old-fashioned-welsh-rarebit\n\nProcedure:\n1. Grate 250g well-flavoured mature Cheddar cheese (you can also use a good Double Gloucester or Shropshire) and put into a heavy-based medium-sized pan (preferably nonstick) with 10g butter, 1 tsp mustard (English, wholegrain or Dijon), plenty of ground black pepper and 4 tbsp ale or semi-skimmed milk.\n2. Set over a low heat and melt gently, stirring, until smooth and thick.\n3. Spoon onto 4 thick slices of toast and grill until browned and bubbling.\n4. Serve with grilled mushrooms and a leaf salad for a light lunch or snack.\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 1 because the cheese mixture needs to be spooned onto the toast before grilling.\" \n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 0, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 5 because the stew needs to be served piping hot, which can only happen after the stew is fully cooked in Step 5.\"\n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 1, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 to ensure that the stew is seasoned well with salt and pepper after all the ingredients have been cooked.\" \n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 2, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 1?", "why_question": "Why must Step 4 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 4 happen after Step 1?\n2. Why must Step 4 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 4 does not have to happen after Step 1, as the order of adding the remaining potatoes and carrots can be adjusted based on cooking time and desired texture.\"\n}", "relevant_step_nos": [4, 1]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 3, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to cook the lamb and remove any excess fat before adding the other ingredients.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 4, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 1 because the lamb needs to be cooked before adding the onion and potatoes.\"\n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 5, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 1?", "why_question": "Why must Step 6 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 6 happen after Step 1?\n2. Why must Step 6 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 does not have to happen after Step 1, as seasoning can be added at any point during the cooking process.\"\n}", "relevant_step_nos": [6, 1]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 6, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 2?", "why_question": "Why must Step 7 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 7 happen after Step 2?\n2. Why must Step 7 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 does not have to happen after Step 2, as it can be done at the end of the cooking process to add fresh flavor to the stew.\"\n}", "relevant_step_nos": [7, 2]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 7, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 2 to ensure that the stew is properly seasoned and flavored before serving.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 8, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 4?", "why_question": "Why must Step 6 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 6 happen after Step 4?\n2. Why must Step 6 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 4 to ensure that the stew is seasoned well with salt and pepper after all the ingredients have been added and cooked.\" \n}", "relevant_step_nos": [6, 4]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 9, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 6 because the seasoning and addition of fresh parsley or chopped leeks should be the final touches before serving the stew.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 10, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 3 to ensure that the stew is seasoned and flavored properly before serving.\"\n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 11, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 3 because the potatoes need to be partially cooked and broken up before adding the remaining potatoes and carrots.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 0, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 1?", "why_question": "Why must Step 10 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 10 happen after Step 1?\n2. Why must Step 10 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 must happen after Step 9, not Step 1, as it involves adding the tuna and parsley to the dish, which is the final step in the procedure.\"\n}", "relevant_step_nos": [10, 1]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 1, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 1?", "why_question": "Why must Step 6 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 6 happen after Step 1?\n2. Why must Step 6 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 must happen after Step 3, not Step 1.\"\n}", "relevant_step_nos": [6, 1]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 2, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 3 because the mushrooms need to release their juices before adding the other ingredients to create the desired flavors and textures.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 3, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 3?", "why_question": "Why must Step 8 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 8 happen after Step 3?\n2. Why must Step 8 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 must happen after Step 9, not Step 3.\"\n}", "relevant_step_nos": [8, 3]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 4, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 3 because the mushrooms need to be cooked and coated with the sauce before adding the other ingredients.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 5, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 7?", "why_question": "Why must Step 10 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 10 happen after Step 7?\n2. Why must Step 10 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 7 because the peppers need to be mixed into the mushrooms before adding the tuna and parsley.\"\n}", "relevant_step_nos": [10, 7]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 6, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 7?", "why_question": "Why must Step 9 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 9 happen after Step 7?\n2. Why must Step 9 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 7 because the tuna needs to be mixed with the peppers and mushrooms before serving.\"\n}", "relevant_step_nos": [9, 7]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 7, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 4 because the mushrooms need to be cooked and coated with the sauce before adding the tuna and parsley.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 8, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 4 because the mushrooms need to release their juices before adding the other ingredients to create the desired flavors and textures.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 9, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 4 because the mushrooms need to be cooked and coated with the sauce before adding the roasted red peppers.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 10, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 8 because it involves mixing the peppers into the mushrooms before adding the tuna and parsley.\"\n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 11, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 8?", "why_question": "Why must Step 10 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 10 happen after Step 8?\n2. Why must Step 10 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 8 because it involves adding the tuna and parsley to the cooked peppers and mushrooms, and then serving the dish immediately.\" \n}", "relevant_step_nos": [10, 8]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 12, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 2 because the mushrooms need to release their juices before adding the other ingredients.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 13, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure the mushrooms are evenly coated with the sauce.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 14, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 2 because the mushrooms need to be added to the pan after the onions have softened.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 15, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the mushrooms need to give up their juices before adding the other ingredients.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 16, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 9?", "why_question": "Why must Step 10 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 10 happen after Step 9?\n2. Why must Step 10 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 9 because it involves serving the dish immediately after adding the tuna and parsley.\"\n}", "relevant_step_nos": [10, 9]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 17, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 5?", "why_question": "Why must Step 10 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 10 happen after Step 5?\n2. Why must Step 10 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 must happen after Step 9, not Step 5, because the tuna is added at the end of the procedure.\" \n}", "relevant_step_nos": [10, 5]}
{"plan_idx": 36, "title": "rouille", "question_idx": 0, "steps": ["In a mortar, crush the garlic to a fine mash.", "Transfer to a mixing bowl.", "Add the egg yolks, and beat into a mayonnaise while adding the olive oil.", "When the sauce is thick and smooth, stir in the cayenne pepper and saffron."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: rouille\n\nProcedure:\n1. In a mortar, crush the garlic to a fine mash.\n2. Transfer to a mixing bowl.\n3. Add the egg yolks, and beat into a mayonnaise while adding the olive oil.\n4. When the sauce is thick and smooth, stir in the cayenne pepper and saffron.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 3 because the cayenne pepper and saffron are added to the thick and smooth sauce created in Step 3.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 36, "title": "rouille", "question_idx": 1, "steps": ["In a mortar, crush the garlic to a fine mash.", "Transfer to a mixing bowl.", "Add the egg yolks, and beat into a mayonnaise while adding the olive oil.", "When the sauce is thick and smooth, stir in the cayenne pepper and saffron."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 1?", "why_question": "Why must Step 4 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: rouille\n\nProcedure:\n1. In a mortar, crush the garlic to a fine mash.\n2. Transfer to a mixing bowl.\n3. Add the egg yolks, and beat into a mayonnaise while adding the olive oil.\n4. When the sauce is thick and smooth, stir in the cayenne pepper and saffron.\n\n\n1. Must Step 4 happen after Step 1?\n2. Why must Step 4 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 1 because the garlic needs to be crushed and mixed with the other ingredients before adding the cayenne pepper and saffron.\"\n}", "relevant_step_nos": [4, 1]}
{"plan_idx": 36, "title": "rouille", "question_idx": 2, "steps": ["In a mortar, crush the garlic to a fine mash.", "Transfer to a mixing bowl.", "Add the egg yolks, and beat into a mayonnaise while adding the olive oil.", "When the sauce is thick and smooth, stir in the cayenne pepper and saffron."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: rouille\n\nProcedure:\n1. In a mortar, crush the garlic to a fine mash.\n2. Transfer to a mixing bowl.\n3. Add the egg yolks, and beat into a mayonnaise while adding the olive oil.\n4. When the sauce is thick and smooth, stir in the cayenne pepper and saffron.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 2 because the egg yolks need to be added to the crushed garlic in the mixing bowl before beating into a mayonnaise with olive oil.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 37, "title": "sardine spread", "question_idx": 0, "steps": ["Place the sardines in a bowl and roughly mash with a fork.", "Add in all of the remaining ingredients and stir well.", "Enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: sardine spread\n\nProcedure:\n1. Place the sardines in a bowl and roughly mash with a fork.\n2. Add in all of the remaining ingredients and stir well.\n3. Enjoy!\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen after Step 1 because the sardines need to be mashed before adding the remaining ingredients.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 38, "title": "scone-pastry", "question_idx": 0, "steps": ["Sift the flour, baking powder and salt into a bowl.", "Add the butter and rub in until the mixture resembles fine crumbs.", "Make a well in the centre and add the milk.", "Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.", "Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: scone-pastry\n\nProcedure:\n1. Sift the flour, baking powder and salt into a bowl.\n2. Add the butter and rub in until the mixture resembles fine crumbs.\n3. Make a well in the centre and add the milk.\n4. Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.\n5. Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 3 because the dough needs to be formed and then rolled out for use.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 38, "title": "scone-pastry", "question_idx": 1, "steps": ["Sift the flour, baking powder and salt into a bowl.", "Add the butter and rub in until the mixture resembles fine crumbs.", "Make a well in the centre and add the milk.", "Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.", "Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: scone-pastry\n\nProcedure:\n1. Sift the flour, baking powder and salt into a bowl.\n2. Add the butter and rub in until the mixture resembles fine crumbs.\n3. Make a well in the centre and add the milk.\n4. Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.\n5. Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 4 because it involves transferring the dough to a lightly floured surface and kneading it, which can only be done after the dough has been mixed and reached the desired consistency in Step 4.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 38, "title": "scone-pastry", "question_idx": 2, "steps": ["Sift the flour, baking powder and salt into a bowl.", "Add the butter and rub in until the mixture resembles fine crumbs.", "Make a well in the centre and add the milk.", "Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.", "Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required."], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: scone-pastry\n\nProcedure:\n1. Sift the flour, baking powder and salt into a bowl.\n2. Add the butter and rub in until the mixture resembles fine crumbs.\n3. Make a well in the centre and add the milk.\n4. Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.\n5. Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required.\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen after Step 1 because the flour, baking powder, and salt need to be combined before adding the butter and rubbing it in.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 38, "title": "scone-pastry", "question_idx": 3, "steps": ["Sift the flour, baking powder and salt into a bowl.", "Add the butter and rub in until the mixture resembles fine crumbs.", "Make a well in the centre and add the milk.", "Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.", "Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: scone-pastry\n\nProcedure:\n1. Sift the flour, baking powder and salt into a bowl.\n2. Add the butter and rub in until the mixture resembles fine crumbs.\n3. Make a well in the centre and add the milk.\n4. Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.\n5. Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 2 because the well in the centre is made to add the milk and mixing it with the dry ingredients.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 38, "title": "scone-pastry", "question_idx": 4, "steps": ["Sift the flour, baking powder and salt into a bowl.", "Add the butter and rub in until the mixture resembles fine crumbs.", "Make a well in the centre and add the milk.", "Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.", "Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: scone-pastry\n\nProcedure:\n1. Sift the flour, baking powder and salt into a bowl.\n2. Add the butter and rub in until the mixture resembles fine crumbs.\n3. Make a well in the centre and add the milk.\n4. Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.\n5. Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 2 because the dough needs to be formed and kneaded before it is rolled out and used.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 0, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the gravy granules need to be added after the corned beef has been simmered to ensure proper flavor and consistency.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 1, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 2 because the stew needs to simmer and thicken before it is ready to serve.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 2, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 5 can happen after Step 4, as it is the addition of gravy granules that thickens the stew.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 3, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 is the final step to complete the dish and make it ready to serve.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 4, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 1?", "why_question": "Why must Step 5 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 5 happen after Step 1?\n2. Why must Step 5 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 5 must happen after Step 4 to allow the gravy granules to thicken the stew.\"\n}", "relevant_step_nos": [5, 1]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 5, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 1?", "why_question": "Why must Step 6 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 6 happen after Step 1?\n2. Why must Step 6 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 1 because the stew needs to be cooked and thickened before serving.\"\n}", "relevant_step_nos": [6, 1]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 0, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 must happen after Step 6, not Step 4, because the cake needs to be baked before checking if it's done.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 1, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 1?", "why_question": "Why must Step 10 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 10 happen after Step 1?\n2. Why must Step 10 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 can happen at any time after the cupcakes have cooled fully.\"\n}", "relevant_step_nos": [10, 1]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 2, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 5?", "why_question": "Why must Step 9 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 9 happen after Step 5?\n2. Why must Step 9 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cupcakes to set and cool down before decorating.\"\n}", "relevant_step_nos": [9, 5]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 3, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 5 because Step 5 involves mixing the flour into the batter, and Step 8 checks if the cake is fully baked.\" \n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 4, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 7?", "why_question": "Why must Step 10 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 10 happen after Step 7?\n2. Why must Step 10 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 7 because the cupcakes need to cool down before they can be decorated.\"\n}", "relevant_step_nos": [10, 7]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 5, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 2?", "why_question": "Why must Step 9 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 9 happen after Step 2?\n2. Why must Step 9 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 must happen after Step 7, not Step 2.\"\n}", "relevant_step_nos": [9, 2]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 6, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 3 to ensure that the flour is gradually added and evenly mixed into the batter.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 7, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 3 because the batter needs to be prepared before spooning it into the muffin tin.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 8, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 6 because the batter needs to be spooned into the muffin tin before baking.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 0, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 6 because it involves the cooking process and the time required for the chicken to become tender and the sauce to reduce.\" \n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 1, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 7?", "why_question": "Why must Step 8 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 8 happen after Step 7?\n2. Why must Step 8 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the cardamom pods are removed before serving the butter chicken.\"\n}", "relevant_step_nos": [8, 7]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 2, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 involves transferring the mixture to the slow cooker, so it must happen after all the ingredients have been combined in the frying pan in Step 4.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 3, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 1?", "why_question": "Why must Step 4 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 4 happen after Step 1?\n2. Why must Step 4 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 1 because the chicken, onion, and garlic need to be cooked before adding the curry powder, curry paste, tandoori masala, garam masala, and tomato puree.\"\n}", "relevant_step_nos": [4, 1]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 4, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 1?", "why_question": "Why must Step 7 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 7 happen after Step 1?\n2. Why must Step 7 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 must happen after Step 5, not Step 1.\"\n}", "relevant_step_nos": [7, 1]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 5, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 1 because the chicken, onion, and garlic need to be cooked in the heated butter and vegetable oil.\"\n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 6, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 1?", "why_question": "Why must Step 6 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 6 happen after Step 1?\n2. Why must Step 6 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 can happen at any point before the slow cooker is turned on.\"\n}", "relevant_step_nos": [6, 1]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 7, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 3 because the chicken and other ingredients need to cook in the slow cooker for several hours to become tender and for the sauce to reduce.\" \n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 8, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 involves transferring the mixture to the slow cooker, which can only happen after the initial cooking and stirring in the frying pan in Step 3.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 9, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 3 to ensure that the onion has softened and turned translucent before adding the spices and tomato puree.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 10, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 5 because it involves cooking the mixture in the slow cooker for the specified duration to ensure the chicken is tender and the sauce has the desired consistency.\"\n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 11, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because seasoning the mixture with salt should be done after all the ingredients have been combined in the slow cooker.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 12, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 involves transferring the mixture to the slow cooker, which can only happen after the chicken, onion, and garlic have been cooked in the frying pan.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 0, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 6 because it involves pouring the dressing over the salad, which includes the grapefruit segments and smoked salmon.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 1, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 2?", "why_question": "Why must Step 8 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 8 happen after Step 2?\n2. Why must Step 8 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 can happen after Step 6, as it is not directly dependent on Step 2.\"\n}", "relevant_step_nos": [8, 2]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 2, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 2?", "why_question": "Why must Step 7 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 7 happen after Step 2?\n2. Why must Step 7 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 can happen after Step 2, but it does not have to happen immediately after Step 2. It can happen at any point before serving the salad.\"\n}", "relevant_step_nos": [7, 2]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 3, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 2 because the grapefruit segments and smoked salmon are added to the salad after the grapefruits have been prepared and segmented.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 4, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 1?", "why_question": "Why must Step 7 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 7 happen after Step 1?\n2. Why must Step 7 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 does not need to happen after Step 1, as it can be done at any point before serving the salad.\"\n}", "relevant_step_nos": [7, 1]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 5, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 4 because the grapefruit juice from the bowl is used to make the dressing for the salad.\" \n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 43, "title": "spanish-rabbit-stew", "question_idx": 0, "steps": ["After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?", ").", "I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.", "Pour in half pint of white wine and half pint of water.", "You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.", "Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.", "Turn your slow cooker on to low power, let it simmer for about 5 hours.", "If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!", "Flavour with salt and pepper to taste.", "Cook the pasta according to the packet instructions, and serve with pasta.", "Most importantly, enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 4?", "why_question": "Why must Step 10 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spanish-rabbit-stew\n\nProcedure:\n1. After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?\n2. ).\n3. I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.\n4. Pour in half pint of white wine and half pint of water.\n5. You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.\n6. Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.\n7. Turn your slow cooker on to low power, let it simmer for about 5 hours.\n8. If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!\n9. Flavour with salt and pepper to taste.\n10. Cook the pasta according to the packet instructions, and serve with pasta.\n11. Most importantly, enjoy!\n\n\n1. Must Step 10 happen after Step 4?\n2. Why must Step 10 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 does not have to happen after Step 4, as it can happen at any time before serving the stew.\"\n}", "relevant_step_nos": [10, 4]}
{"plan_idx": 43, "title": "spanish-rabbit-stew", "question_idx": 1, "steps": ["After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?", ").", "I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.", "Pour in half pint of white wine and half pint of water.", "You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.", "Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.", "Turn your slow cooker on to low power, let it simmer for about 5 hours.", "If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!", "Flavour with salt and pepper to taste.", "Cook the pasta according to the packet instructions, and serve with pasta.", "Most importantly, enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 1?", "why_question": "Why must Step 10 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spanish-rabbit-stew\n\nProcedure:\n1. After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?\n2. ).\n3. I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.\n4. Pour in half pint of white wine and half pint of water.\n5. You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.\n6. Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.\n7. Turn your slow cooker on to low power, let it simmer for about 5 hours.\n8. If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!\n9. Flavour with salt and pepper to taste.\n10. Cook the pasta according to the packet instructions, and serve with pasta.\n11. Most importantly, enjoy!\n\n\n1. Must Step 10 happen after Step 1?\n2. Why must Step 10 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 can happen after Step 1, but it is not a requirement for the stew to be prepared.\" \n}", "relevant_step_nos": [10, 1]}
{"plan_idx": 43, "title": "spanish-rabbit-stew", "question_idx": 2, "steps": ["After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?", ").", "I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.", "Pour in half pint of white wine and half pint of water.", "You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.", "Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.", "Turn your slow cooker on to low power, let it simmer for about 5 hours.", "If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!", "Flavour with salt and pepper to taste.", "Cook the pasta according to the packet instructions, and serve with pasta.", "Most importantly, enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 1?", "why_question": "Why must Step 4 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spanish-rabbit-stew\n\nProcedure:\n1. After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?\n2. ).\n3. I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.\n4. Pour in half pint of white wine and half pint of water.\n5. You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.\n6. Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.\n7. Turn your slow cooker on to low power, let it simmer for about 5 hours.\n8. If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!\n9. Flavour with salt and pepper to taste.\n10. Cook the pasta according to the packet instructions, and serve with pasta.\n11. Most importantly, enjoy!\n\n\n1. Must Step 4 happen after Step 1?\n2. Why must Step 4 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 4 does not have to happen after Step 1, as the order of adding the wine and water can be adjusted based on the amount of ingredients in the slow cooker.\"\n}", "relevant_step_nos": [4, 1]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 0, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 2?", "why_question": "Why must Step 15 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 15 happen after Step 2?\n2. Why must Step 15 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 15 must happen after Step 2 because Step 15 involves seasoning and adding additional ingredients to the soup, which can only be done after the soup has been pureed in Step 13.\"\n}", "relevant_step_nos": [15, 2]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 1, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 2?", "why_question": "Why must Step 17 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 17 happen after Step 2?\n2. Why must Step 17 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 17 must happen after Step 16, not Step 2.\"\n}", "relevant_step_nos": [17, 2]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 2, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the squash halves need to be baked in the oven before they can be pierced with a fork to check for doneness.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 3, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 can happen after Step 4, not necessarily after Step 2.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 4, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 2?", "why_question": "Why must Step 12 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 12 happen after Step 2?\n2. Why must Step 12 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 must happen after Step 4, not Step 2, because the squash needs to be baked and then mashed with the potatoes before pureeing the soup.\"\n}", "relevant_step_nos": [12, 2]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 5, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 4?", "why_question": "Why must Step 15 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 15 happen after Step 4?\n2. Why must Step 15 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 15 can happen after Step 4, but it does not have to happen immediately after Step 4.\"\n}", "relevant_step_nos": [15, 4]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 6, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 4?", "why_question": "Why must Step 16 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 16 happen after Step 4?\n2. Why must Step 16 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 4 because the squash needs to be baked and prepared before the soup is heated through.\"\n}", "relevant_step_nos": [16, 4]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 7, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 4 because the squash needs to cool slightly before removing the peel.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 8, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 4?", "why_question": "Why must Step 13 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 13 happen after Step 4?\n2. Why must Step 13 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 4 because the squash needs to be baked and then cooled before it can be pureed.\" \n}", "relevant_step_nos": [13, 4]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 9, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 4?", "why_question": "Why must Step 14 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 14 happen after Step 4?\n2. Why must Step 14 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 can happen after Step 4, but it does not have to happen immediately after Step 4.\"\n}", "relevant_step_nos": [14, 4]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 10, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 4?", "why_question": "Why must Step 17 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 17 happen after Step 4?\n2. Why must Step 17 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 17 can happen after Step 4, but it does not have to happen immediately after Step 4.\"\n}", "relevant_step_nos": [17, 4]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 11, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 4?", "why_question": "Why must Step 12 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 12 happen after Step 4?\n2. Why must Step 12 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 requires the squash to be baked and then cooled before removing the peel and mashing with the potatoes.\"\n}", "relevant_step_nos": [12, 4]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 12, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 7?", "why_question": "Why must Step 13 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 13 happen after Step 7?\n2. Why must Step 13 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 7 because the squash needs to be added to the pot after the other ingredients have been sauteed and the chicken stock has been added.\"\n}", "relevant_step_nos": [13, 7]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 13, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 7?", "why_question": "Why must Step 15 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 15 happen after Step 7?\n2. Why must Step 15 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 7 because the soup needs to be seasoned and have the sherry, cream, and milk added after the initial ingredients have been cooked and pureed.\"\n}", "relevant_step_nos": [15, 7]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 14, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 7?", "why_question": "Why must Step 9 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 9 happen after Step 7?\n2. Why must Step 9 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 7 because the chicken stock needs to be added to the pot after saut\u00e9ing the onion, leek, and garlic.\"\n}", "relevant_step_nos": [9, 7]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 15, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 7?", "why_question": "Why must Step 16 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 16 happen after Step 7?\n2. Why must Step 16 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 7 because the soup needs to be heated through before serving.\"\n}", "relevant_step_nos": [16, 7]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 16, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 7?", "why_question": "Why must Step 11 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 11 happen after Step 7?\n2. Why must Step 11 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 7 because the potatoes need to be cooked before adding the squash.\"\n}", "relevant_step_nos": [11, 7]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 17, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 3?", "why_question": "Why must Step 16 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 16 happen after Step 3?\n2. Why must Step 16 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 16 must happen after Step 15, not Step 3.\"\n}", "relevant_step_nos": [16, 3]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 18, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 3 because the squash needs to be baked before the flesh can be easily pierced with a fork.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 19, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 3?", "why_question": "Why must Step 12 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 12 happen after Step 3?\n2. Why must Step 12 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 requires the squash to be baked and then cooled before removing the peel.\"\n}", "relevant_step_nos": [12, 3]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 20, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 3?", "why_question": "Why must Step 15 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 15 happen after Step 3?\n2. Why must Step 15 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 15 must happen after Step 3 because the squash needs to be cooked and mashed before adding the spices and other ingredients.\"\n}", "relevant_step_nos": [15, 3]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 21, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 9?", "why_question": "Why must Step 11 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 11 happen after Step 9?\n2. Why must Step 11 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 9 because the potatoes need to cook in the chicken stock before adding the squash.\"\n}", "relevant_step_nos": [11, 9]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 22, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 9?", "why_question": "Why must Step 14 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 14 happen after Step 9?\n2. Why must Step 14 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 must happen after Step 13, not Step 9, because it involves seasoning the soup and adding sherry, cream, and milk, which are the final steps in the procedure.\"\n}", "relevant_step_nos": [14, 9]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 23, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 11?", "why_question": "Why must Step 12 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 12 happen after Step 11?\n2. Why must Step 12 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 requires the squash to be added to the potatoes, which are cooked in Step 11.\"\n}", "relevant_step_nos": [12, 11]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 24, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 11?", "why_question": "Why must Step 16 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 16 happen after Step 11?\n2. Why must Step 16 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 11 because the soup needs to be heated through before serving.\"\n}", "relevant_step_nos": [16, 11]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 25, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 11?", "why_question": "Why must Step 14 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 14 happen after Step 11?\n2. Why must Step 14 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 can happen after Step 13, as it involves seasoning and adding additional ingredients to the soup after it has been pureed.\"\n}", "relevant_step_nos": [14, 11]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 26, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 11?", "why_question": "Why must Step 17 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 17 happen after Step 11?\n2. Why must Step 17 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 17 can happen after Step 11, but it does not have to happen immediately after Step 11.\"\n}", "relevant_step_nos": [17, 11]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 27, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 14?", "why_question": "Why must Step 15 happen after Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 15 happen after Step 14?\n2. Why must Step 15 happen after Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 14 because the soup needs to be seasoned and have the sherry, cream, and milk added before it is heated through.\"\n}", "relevant_step_nos": [15, 14]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 28, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 14?", "why_question": "Why must Step 17 happen after Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 17 happen after Step 14?\n2. Why must Step 17 happen after Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 14 to complete the soup and serve it in bowls.\"\n}", "relevant_step_nos": [17, 14]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 29, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 10?", "why_question": "Why must Step 14 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 14 happen after Step 10?\n2. Why must Step 14 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 must happen after Step 13 to season the soup and add the sherry, cream, and milk.\"\n}", "relevant_step_nos": [14, 10]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 30, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 12?", "why_question": "Why must Step 15 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 15 happen after Step 12?\n2. Why must Step 15 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 12 because the seasoning and addition of sherry, cream, and milk should be done after the soup has been pureed and returned to the pot.\"\n}", "relevant_step_nos": [15, 12]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 31, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 12?", "why_question": "Why must Step 16 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 16 happen after Step 12?\n2. Why must Step 16 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 12 because it involves heating the soup after pureeing it and adding the final seasonings and ingredients.\"\n}", "relevant_step_nos": [16, 12]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 32, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 can only happen after Step 5 because the squash needs to cool slightly before the peel can be removed.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 33, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 16?", "why_question": "Why must Step 17 happen after Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 17 happen after Step 16?\n2. Why must Step 17 happen after Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 16 to ensure that the soup is properly heated through before serving.\"\n}", "relevant_step_nos": [17, 16]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 34, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 1?", "why_question": "Why must Step 12 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 12 happen after Step 1?\n2. Why must Step 12 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 must happen after Step 4, not Step 1.\"\n}", "relevant_step_nos": [12, 1]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 35, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 1?", "why_question": "Why must Step 15 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 15 happen after Step 1?\n2. Why must Step 15 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 15 must happen after Step 1 because it involves seasoning and adding additional ingredients to the soup, which occurs after the soup has been pureed and returned to the pot.\" \n}", "relevant_step_nos": [15, 1]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 36, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 1?", "why_question": "Why must Step 13 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 13 happen after Step 1?\n2. Why must Step 13 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 1 because the squash needs to be baked and cooled before it can be pureed.\"\n}", "relevant_step_nos": [13, 1]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 37, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 17 happen after Step 13?", "why_question": "Why must Step 17 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 17 happen after Step 13?\n2. Why must Step 17 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen after Step 13 because the soup needs to be fully prepared and ready to serve before adding the soured cream topping.\"\n}", "relevant_step_nos": [17, 13]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 38, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 13?", "why_question": "Why must Step 15 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 15 happen after Step 13?\n2. Why must Step 15 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 13 because the seasoning and addition of sherry, cream, and milk should be done after the soup has been pureed and returned to the pot.\"\n}", "relevant_step_nos": [15, 13]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 39, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 16 happen after Step 13?", "why_question": "Why must Step 16 happen after Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 16 happen after Step 13?\n2. Why must Step 16 happen after Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen after Step 13 to ensure that the soup is heated through before serving.\"\n}", "relevant_step_nos": [16, 13]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 40, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 8?", "why_question": "Why must Step 14 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 14 happen after Step 8?\n2. Why must Step 14 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 can happen after Step 8, but it does not have to happen immediately after Step 8.\"\n}", "relevant_step_nos": [14, 8]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 41, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 requires the onion, leek, and garlic to be sauteed before adding the chicken stock.\"\n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 45, "title": "spiced-guava-cake", "question_idx": 0, "steps": ["Preheat oven to 180 C / Gas 4.", "Grease and flour one 23x13cm loaf tin.", "Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.", "In another bowl, mix together guava pulp and juice.", "In another bowl, cream together butter and sugar.", "Add eggs, one at a time.", "Add flour mixture and guava mixture alternately to creamed mixture.", "Pour batter into prepared tin.", "Bake at 180 C / Gas 4 for 30-35 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 7?", "why_question": "Why must Step 9 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-guava-cake\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Grease and flour one 23x13cm loaf tin.\n3. Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.\n4. In another bowl, mix together guava pulp and juice.\n5. In another bowl, cream together butter and sugar.\n6. Add eggs, one at a time.\n7. Add flour mixture and guava mixture alternately to creamed mixture.\n8. Pour batter into prepared tin.\n9. Bake at 180 C / Gas 4 for 30-35 minutes.\n\n\n1. Must Step 9 happen after Step 7?\n2. Why must Step 9 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 7 because pouring the batter into the tin should be the last step before baking.\" \n}", "relevant_step_nos": [9, 7]}
{"plan_idx": 45, "title": "spiced-guava-cake", "question_idx": 1, "steps": ["Preheat oven to 180 C / Gas 4.", "Grease and flour one 23x13cm loaf tin.", "Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.", "In another bowl, mix together guava pulp and juice.", "In another bowl, cream together butter and sugar.", "Add eggs, one at a time.", "Add flour mixture and guava mixture alternately to creamed mixture.", "Pour batter into prepared tin.", "Bake at 180 C / Gas 4 for 30-35 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-guava-cake\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Grease and flour one 23x13cm loaf tin.\n3. Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.\n4. In another bowl, mix together guava pulp and juice.\n5. In another bowl, cream together butter and sugar.\n6. Add eggs, one at a time.\n7. Add flour mixture and guava mixture alternately to creamed mixture.\n8. Pour batter into prepared tin.\n9. Bake at 180 C / Gas 4 for 30-35 minutes.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the flour and guava mixture are evenly incorporated into the creamed mixture.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 45, "title": "spiced-guava-cake", "question_idx": 2, "steps": ["Preheat oven to 180 C / Gas 4.", "Grease and flour one 23x13cm loaf tin.", "Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.", "In another bowl, mix together guava pulp and juice.", "In another bowl, cream together butter and sugar.", "Add eggs, one at a time.", "Add flour mixture and guava mixture alternately to creamed mixture.", "Pour batter into prepared tin.", "Bake at 180 C / Gas 4 for 30-35 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-guava-cake\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Grease and flour one 23x13cm loaf tin.\n3. Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.\n4. In another bowl, mix together guava pulp and juice.\n5. In another bowl, cream together butter and sugar.\n6. Add eggs, one at a time.\n7. Add flour mixture and guava mixture alternately to creamed mixture.\n8. Pour batter into prepared tin.\n9. Bake at 180 C / Gas 4 for 30-35 minutes.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because the eggs need to be added to the creamed butter and sugar before adding the flour and guava mixtures.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 45, "title": "spiced-guava-cake", "question_idx": 3, "steps": ["Preheat oven to 180 C / Gas 4.", "Grease and flour one 23x13cm loaf tin.", "Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.", "In another bowl, mix together guava pulp and juice.", "In another bowl, cream together butter and sugar.", "Add eggs, one at a time.", "Add flour mixture and guava mixture alternately to creamed mixture.", "Pour batter into prepared tin.", "Bake at 180 C / Gas 4 for 30-35 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-guava-cake\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Grease and flour one 23x13cm loaf tin.\n3. Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.\n4. In another bowl, mix together guava pulp and juice.\n5. In another bowl, cream together butter and sugar.\n6. Add eggs, one at a time.\n7. Add flour mixture and guava mixture alternately to creamed mixture.\n8. Pour batter into prepared tin.\n9. Bake at 180 C / Gas 4 for 30-35 minutes.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 4 because the batter needs to be prepared and poured into the tin before baking.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 0, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 9?", "why_question": "Why must Step 13 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 13 happen after Step 9?\n2. Why must Step 13 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 11, not Step 9, to ensure the teabread is still warm when brushed with honey.\"\n}", "relevant_step_nos": [13, 9]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 1, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 9?", "why_question": "Why must Step 10 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 10 happen after Step 9?\n2. Why must Step 10 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 9 because sprinkling with pumpkin seeds should be done after the mixture is placed in the tin and the surface is smoothed.\"\n}", "relevant_step_nos": [10, 9]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 2, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 9?", "why_question": "Why must Step 11 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 11 happen after Step 9?\n2. Why must Step 11 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 9 because the loaf needs to be baked before it can be allowed to cool in the tin.\"\n}", "relevant_step_nos": [11, 9]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 3, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 9?", "why_question": "Why must Step 12 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 12 happen after Step 9?\n2. Why must Step 12 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the loaf to cool slightly and set before removing it from the tin.\"\n}", "relevant_step_nos": [12, 9]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 4, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 5?", "why_question": "Why must Step 13 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 13 happen after Step 5?\n2. Why must Step 13 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 11 to allow the teabread to cool slightly before brushing with honey.\"\n}", "relevant_step_nos": [13, 5]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 5, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 5?", "why_question": "Why must Step 9 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 9 happen after Step 5?\n2. Why must Step 9 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 5 because the pumpkin mixture needs to be added to the tin before smoothing the surface and sprinkling with pumpkin seeds.\"\n}", "relevant_step_nos": [9, 5]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 6, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 5?", "why_question": "Why must Step 14 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 14 happen after Step 5?\n2. Why must Step 14 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 can happen after Step 5, but it is not required to happen immediately after Step 5.\"\n}", "relevant_step_nos": [14, 5]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 7, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 5?", "why_question": "Why must Step 12 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 12 happen after Step 5?\n2. Why must Step 12 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 must happen after Step 11 to allow the loaf to cool before turning it out onto a wire rack.\"\n}", "relevant_step_nos": [12, 5]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 8, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 5?", "why_question": "Why must Step 10 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 10 happen after Step 5?\n2. Why must Step 10 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 5 because it involves sprinkling the pumpkin seeds on top of the mixture before baking.\"\n}", "relevant_step_nos": [10, 5]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 9, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 11?", "why_question": "Why must Step 13 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 13 happen after Step 11?\n2. Why must Step 13 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 11 to ensure the teabread is still warm and the honey can be brushed on easily.\"\n}", "relevant_step_nos": [13, 11]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 10, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 11?", "why_question": "Why must Step 12 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 12 happen after Step 11?\n2. Why must Step 12 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the loaf to cool slightly and set before removing it from the tin.\"\n}", "relevant_step_nos": [12, 11]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 11, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 2?", "why_question": "Why must Step 9 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 9 happen after Step 2?\n2. Why must Step 9 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 2 because the mixture needs to be placed in the prepared tin before baking.\"\n}", "relevant_step_nos": [9, 2]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 12, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 2?", "why_question": "Why must Step 13 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 13 happen after Step 2?\n2. Why must Step 13 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 12 to allow the teabread to cool slightly before brushing with honey.\"\n}", "relevant_step_nos": [13, 2]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 13, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 involves using the ingredients prepared in Step 6 to create the desired texture for the teabread.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 14, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 6?", "why_question": "Why must Step 11 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 11 happen after Step 6?\n2. Why must Step 11 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 6 because the mixture needs to be prepared and placed in the tin before baking.\"\n}", "relevant_step_nos": [11, 6]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 15, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 6?", "why_question": "Why must Step 12 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 12 happen after Step 6?\n2. Why must Step 12 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 6 because the loaf needs to cool in the tin before being turned out onto a wire rack.\"\n}", "relevant_step_nos": [12, 6]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 16, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 6?", "why_question": "Why must Step 13 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 13 happen after Step 6?\n2. Why must Step 13 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 11 to allow the teabread to cool slightly before brushing with honey.\"\n}", "relevant_step_nos": [13, 6]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 17, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 3?", "why_question": "Why must Step 14 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 14 happen after Step 3?\n2. Why must Step 14 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 does not need to happen after Step 3, as it is about serving the teabread and not directly related to the preparation process.\"\n}", "relevant_step_nos": [14, 3]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 18, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 3?", "why_question": "Why must Step 8 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 8 happen after Step 3?\n2. Why must Step 8 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 requires the pumpkin mixture, which is prepared in Step 3.\"\n}", "relevant_step_nos": [8, 3]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 19, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 3?", "why_question": "Why must Step 13 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 13 happen after Step 3?\n2. Why must Step 13 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 11 to allow the teabread to cool slightly before brushing with honey.\"\n}", "relevant_step_nos": [13, 3]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 20, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 3?", "why_question": "Why must Step 12 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 12 happen after Step 3?\n2. Why must Step 12 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the loaf to cool slightly before removing it from the tin.\"\n}", "relevant_step_nos": [12, 3]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 21, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 3?", "why_question": "Why must Step 10 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 10 happen after Step 3?\n2. Why must Step 10 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 happens after Step 3 because the pumpkin seeds are sprinkled on the mixture before baking.\"\n}", "relevant_step_nos": [10, 3]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 22, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 4?", "why_question": "Why must Step 12 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 12 happen after Step 4?\n2. Why must Step 12 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 4 to allow the teabread to cool and set before removing it from the tin.\"\n}", "relevant_step_nos": [12, 4]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 23, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 requires the mashed pumpkin from Step 4 to be added to the mixture.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 24, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 4?", "why_question": "Why must Step 13 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 13 happen after Step 4?\n2. Why must Step 13 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 12 to allow the teabread to cool slightly before brushing with honey.\"\n}", "relevant_step_nos": [13, 4]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 25, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 4?", "why_question": "Why must Step 14 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 14 happen after Step 4?\n2. Why must Step 14 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 can happen after Step 4, but it is not required to happen immediately after Step 4.\"\n}", "relevant_step_nos": [14, 4]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 26, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 8?", "why_question": "Why must Step 12 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 12 happen after Step 8?\n2. Why must Step 12 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the loaf to cool slightly before removing it from the tin.\"\n}", "relevant_step_nos": [12, 8]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 27, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 8?", "why_question": "Why must Step 10 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 10 happen after Step 8?\n2. Why must Step 10 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 8 because it instructs to sprinkle the mixture with pumpkin seeds, which can only be done after the mixture is in the tin from Step 8.\"\n}", "relevant_step_nos": [10, 8]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 28, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 requires the mixture from Step 8 to be placed in the tin before baking.\"\n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 29, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 8?", "why_question": "Why must Step 11 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 11 happen after Step 8?\n2. Why must Step 11 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 8 because the mixture needs to be baked after it is evenly mixed and placed in the tin.\"\n}", "relevant_step_nos": [11, 8]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 30, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 10?", "why_question": "Why must Step 12 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 12 happen after Step 10?\n2. Why must Step 12 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the loaf to cool slightly before removing it from the tin.\"\n}", "relevant_step_nos": [12, 10]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 31, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 1?", "why_question": "Why must Step 12 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 12 happen after Step 1?\n2. Why must Step 12 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 1 because the teabread needs to cool in the tin for 10 minutes before being turned out onto a wire rack.\"\n}", "relevant_step_nos": [12, 1]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 32, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 1?", "why_question": "Why must Step 14 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 14 happen after Step 1?\n2. Why must Step 14 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 must happen after Step 11 to allow the teabread to cool and be ready for serving.\"\n}", "relevant_step_nos": [14, 1]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 33, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 1?", "why_question": "Why must Step 13 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 13 happen after Step 1?\n2. Why must Step 13 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 11 to allow the teabread to cool slightly before brushing with honey.\"\n}", "relevant_step_nos": [13, 1]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 34, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 1?", "why_question": "Why must Step 11 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 11 happen after Step 1?\n2. Why must Step 11 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 1 because the oven needs to be preheated before baking the teabread.\"\n}", "relevant_step_nos": [11, 1]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 35, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 7?", "why_question": "Why must Step 9 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 9 happen after Step 7?\n2. Why must Step 9 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 can only happen after Step 7 because the pumpkin mixture needs to be added to the flour mixture before it can be tipped into the tin.\"\n}", "relevant_step_nos": [9, 7]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 36, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 7?", "why_question": "Why must Step 11 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 11 happen after Step 7?\n2. Why must Step 11 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 7 because the mixture needs to be prepared and placed in the tin before baking.\"\n}", "relevant_step_nos": [11, 7]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 37, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 7?", "why_question": "Why must Step 12 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 12 happen after Step 7?\n2. Why must Step 12 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 7 because the loaf needs to cool in the tin before being turned out onto a wire rack.\"\n}", "relevant_step_nos": [12, 7]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 0, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 8?", "why_question": "Why must Step 10 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 10 happen after Step 8?\n2. Why must Step 10 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 8 because it instructs to serve the dish immediately after tossing with the anchovy mixture and chilli flakes.\"\n}", "relevant_step_nos": [10, 8]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 1, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 6 because the linguine needs to be cooked before it can be drained and tossed with the anchovy mixture.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 2, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 5?", "why_question": "Why must Step 10 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 10 happen after Step 5?\n2. Why must Step 10 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 5 because the pasta needs to be cooked and ready to be tossed with the anchovy mixture before serving.\"\n}", "relevant_step_nos": [10, 5]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 3, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 7?", "why_question": "Why must Step 10 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 10 happen after Step 7?\n2. Why must Step 10 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 7 because the pasta needs to be cooked and drained before being tossed with the anchovy mixture and served immediately.\"\n}", "relevant_step_nos": [10, 7]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 4, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 2?", "why_question": "Why must Step 10 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 10 happen after Step 2?\n2. Why must Step 10 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 can happen after Step 8, as it is the final step in the procedure.\"\n}", "relevant_step_nos": [10, 2]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 5, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 requires the anchovies and water to be added to the cooked vegetables from Step 2.\"\n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 6, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 2?", "why_question": "Why must Step 8 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 8 happen after Step 2?\n2. Why must Step 8 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 must happen after Step 6, not Step 2, as it involves tossing the linguine with the anchovy mixture and chilli flakes after the pasta has been cooked.\"\n}", "relevant_step_nos": [8, 2]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 7, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 9?", "why_question": "Why must Step 10 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 10 happen after Step 9?\n2. Why must Step 10 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 is the final step in the procedure, so it must happen after Step 9 to ensure the dish is served immediately after seasoning with black pepper.\"\n}", "relevant_step_nos": [10, 9]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 8, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 requires the vegetables to be cooked for a longer period of time after adding the anchovies and water in Step 3.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 0, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 4?", "why_question": "Why must Step 6 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 6 happen after Step 4?\n2. Why must Step 6 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 4 to ensure the strawberries are properly infused with the Pimm's flavor before garnishing with mint leaves.\"\n}", "relevant_step_nos": [6, 4]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 1, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 can happen at any point after Step 4, it does not have to be directly after Step 2.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 2, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 allows the strawberries to absorb the Pimm's flavor after being coated with sugar in Step 2.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 3, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 1?", "why_question": "Why must Step 6 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 6 happen after Step 1?\n2. Why must Step 6 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 can happen at any point after Step 1, it does not have to be immediately after Step 1.\"\n}", "relevant_step_nos": [6, 1]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 4, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 1?", "why_question": "Why must Step 5 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 5 happen after Step 1?\n2. Why must Step 5 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 allows the strawberries to marinate in the Pimm's and develop flavor after being coated with sugar in Step 2.\"\n}", "relevant_step_nos": [5, 1]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 5, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 3 to allow the strawberries to absorb the sugar before adding the Pimm's.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 6, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the strawberries to absorb the Pimm's flavor after being coated with sugar.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 7, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 does not have to happen after Step 3, as it is an optional garnish and can be added at any time before serving.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 8, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 to ensure that the strawberries are properly infused with the Pimm's flavor before garnishing with mint leaves.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 0, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the layers are properly assembled and the strawberries and grated chocolate are evenly distributed.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 1, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 6?", "why_question": "Why must Step 10 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 10 happen after Step 6?\n2. Why must Step 10 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 6 to allow the trifle to set and the flavors to meld together in the fridge.\"\n}", "relevant_step_nos": [10, 6]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 2, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 5 to ensure the layers are properly assembled in the trifle.\"\n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 3, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 4?", "why_question": "Why must Step 6 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 6 happen after Step 4?\n2. Why must Step 6 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 4 because the strawberries are used in Step 6 and need to be prepared before they can be spread over the sponge fingers.\"\n}", "relevant_step_nos": [6, 4]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 4, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 4?", "why_question": "Why must Step 7 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 7 happen after Step 4?\n2. Why must Step 7 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 4 because the soaked sponge fingers need to be layered with the strawberries, grated chocolate, and custard mixture.\"\n}", "relevant_step_nos": [7, 4]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 5, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 4 because the soaked sponge fingers need to be lined in the glass bowl before adding the other layers.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 6, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 3 because the custard mixture needs to be prepared before layering it in the trifle.\"\n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 7, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 1?", "why_question": "Why must Step 10 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 10 happen after Step 1?\n2. Why must Step 10 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the trifle to set and the flavors to meld together.\"\n}", "relevant_step_nos": [10, 1]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 8, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 7?", "why_question": "Why must Step 9 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 9 happen after Step 7?\n2. Why must Step 9 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 7 to ensure the strawberries are evenly distributed and visible on top of the trifle.\"\n}", "relevant_step_nos": [9, 7]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 0, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 2 because the tomatoes and apples need to be prepared before being used in the mixture for the gammon steaks.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 1, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 2?", "why_question": "Why must Step 7 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 7 happen after Step 2?\n2. Why must Step 7 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 2 because the gammon steaks need to be prepared and filled with the mixture before being placed in the casserole dish.\"\n}", "relevant_step_nos": [7, 2]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 2, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The tomatoes need to be softened before the skins can be removed and chopped.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 3, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 3?", "why_question": "Why must Step 8 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 8 happen after Step 3?\n2. Why must Step 8 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 3 because the onions, tomatoes, and apples need to be prepared and placed in the casserole dish before pouring the cider over the steaks.\"\n}", "relevant_step_nos": [8, 3]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 4, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 requires the onions to be chopped, which happens in Step 3.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 5, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 3 because the onions, tomatoes, and apples need to be prepared and placed in the casserole dish before the gammon steaks are wrapped and placed in the dish.\"\n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 6, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 6 because the cider needs to be poured over the gammon steaks before they are covered and cooked in the oven.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 7, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 4 because the tomatoes need to be prepared and placed in the casserole dish before cooking for 1 hour.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 0, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 1?", "why_question": "Why must Step 5 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 5 happen after Step 1?\n2. Why must Step 5 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the flavors to meld together before serving.\"\n}", "relevant_step_nos": [5, 1]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 1, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 1?", "why_question": "Why must Step 4 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 4 happen after Step 1?\n2. Why must Step 4 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 4 can happen at any point after Step 2, it does not have to happen after Step 1.\"\n}", "relevant_step_nos": [4, 1]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 2, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 1 because the tomatoes need to be sliced and placed on the plate before the basil leaves are scattered over them.\"\n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 3, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen after Step 1 because the serving plate needs to be prepared before the tomatoes are sliced and scattered onto it.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 4, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 3?", "why_question": "Why must Step 4 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 4 happen after Step 3?\n2. Why must Step 4 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Drizzling the vinegar and oil after scattering the basil leaves ensures that the flavors are evenly distributed and the salad is properly seasoned.\"\n}", "relevant_step_nos": [4, 3]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 5, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 allows the flavors to meld together and develop while the salad is left at room temperature.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 6, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 2?", "why_question": "Why must Step 3 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 3 happen after Step 2?\n2. Why must Step 3 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 2 because the basil leaves are scattered over the sliced tomatoes.\" \n}", "relevant_step_nos": [3, 2]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 7, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 2?", "why_question": "Why must Step 5 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 5 happen after Step 2?\n2. Why must Step 5 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 allows the flavors to meld together and develop while the salad is left at room temperature.\"\n}", "relevant_step_nos": [5, 2]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 0, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 3 because the tortellini needs to be fried and cooked before serving.\"\n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 1, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 3?", "why_question": "Why must Step 5 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 5 happen after Step 3?\n2. Why must Step 5 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 3 because the almond butter is added during the frying process in Step 3.\"\n}", "relevant_step_nos": [5, 3]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 2, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 3 happen after Step 1?", "why_question": "Why must Step 3 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 3 happen after Step 1?\n2. Why must Step 3 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen after Step 1 because the tortellini needs to be cooked before it can be fried.\"\n}", "relevant_step_nos": [3, 1]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 3, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 1?", "why_question": "Why must Step 5 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 5 happen after Step 1?\n2. Why must Step 5 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 5 can happen after Step 3, not necessarily after Step 1.\"\n}", "relevant_step_nos": [5, 1]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 4, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 1?", "why_question": "Why must Step 6 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 6 happen after Step 1?\n2. Why must Step 6 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 must happen after Step 3, not Step 1.\"\n}", "relevant_step_nos": [6, 1]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 5, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 2?", "why_question": "Why must Step 7 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 7 happen after Step 2?\n2. Why must Step 7 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 2 because the tortellini needs to be cooked and fried before serving.\"\n}", "relevant_step_nos": [7, 2]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 6, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 6?", "why_question": "Why must Step 7 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 7 happen after Step 6?\n2. Why must Step 7 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen after Step 6 because it involves serving the tortellini after it has been fried to golden brown.\"\n}", "relevant_step_nos": [7, 6]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 0, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 5?", "why_question": "Why must Step 11 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 11 happen after Step 5?\n2. Why must Step 11 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 must happen after Step 5 because the cream mixture needs to be prepared before arranging the vegetables in the terrine.\"\n}", "relevant_step_nos": [11, 5]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 1, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 requires the cream and yogurt to be brought to a boil in order to add the chopped sorrel and gelatin.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 2, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 5?", "why_question": "Why must Step 12 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 12 happen after Step 5?\n2. Why must Step 12 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 5 because the cream mixture needs to be prepared before pouring it over the arranged vegetables.\"\n}", "relevant_step_nos": [12, 5]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 3, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 5?", "why_question": "Why must Step 13 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 13 happen after Step 5?\n2. Why must Step 13 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 5 because the cream mixture needs to be refrigerated to become firm before the terrine can be served.\"\n}", "relevant_step_nos": [13, 5]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 4, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 5?", "why_question": "Why must Step 8 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 8 happen after Step 5?\n2. Why must Step 8 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The cream mixture needs to be brought to a boil before adding the chopped sorrel and gelatin in Step 8.\"\n}", "relevant_step_nos": [8, 5]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 5, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 5?", "why_question": "Why must Step 14 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 14 happen after Step 5?\n2. Why must Step 14 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 is necessary to release the terrine from the mould after it has set in the refrigerator.\"\n}", "relevant_step_nos": [14, 5]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 6, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 5?", "why_question": "Why must Step 15 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 15 happen after Step 5?\n2. Why must Step 15 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 5 because the terrine needs to be refrigerated for several hours until firm before serving chilled.\"\n}", "relevant_step_nos": [15, 5]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 7, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 10?", "why_question": "Why must Step 15 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 15 happen after Step 10?\n2. Why must Step 15 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 10 because the terrine needs to be refrigerated for several hours until firm before serving chilled.\"\n}", "relevant_step_nos": [15, 10]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 8, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 10?", "why_question": "Why must Step 11 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 11 happen after Step 10?\n2. Why must Step 11 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 10 because it involves arranging the vegetables in the mould, which can only be done after the initial layer of cream mixture has been poured and refrigerated.\"\n}", "relevant_step_nos": [11, 10]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 9, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 6?", "why_question": "Why must Step 15 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 15 happen after Step 6?\n2. Why must Step 15 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 6 because the terrine needs to be refrigerated for several hours until firm before serving chilled.\"\n}", "relevant_step_nos": [15, 6]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 10, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 6?", "why_question": "Why must Step 13 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 13 happen after Step 6?\n2. Why must Step 13 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 6 to allow the terrine to set and firm up in the refrigerator.\"\n}", "relevant_step_nos": [13, 6]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 11, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 6?", "why_question": "Why must Step 14 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 14 happen after Step 6?\n2. Why must Step 14 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen after Step 6 because the terrine needs to be refrigerated for several hours until firm before it can be served chilled.\"\n}", "relevant_step_nos": [14, 6]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 12, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 9 does not need to happen after Step 4, as they are independent of each other.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 13, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 4?", "why_question": "Why must Step 11 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 11 happen after Step 4?\n2. Why must Step 11 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 4 because the chopped sorrel needs to be cooked and set aside before arranging the vegetables in the terrine.\"\n}", "relevant_step_nos": [11, 4]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 14, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 4?", "why_question": "Why must Step 14 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 14 happen after Step 4?\n2. Why must Step 14 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 does not need to happen after Step 4, as it is not directly related to the cooking or preparation of the terrine.\"\n}", "relevant_step_nos": [14, 4]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 15, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 4?", "why_question": "Why must Step 13 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 13 happen after Step 4?\n2. Why must Step 13 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 4 because Step 13 involves refrigerating the terrine for several hours until firm, while Step 4 involves cooking the sorrel and setting it aside.\"\n}", "relevant_step_nos": [13, 4]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 16, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 11?", "why_question": "Why must Step 14 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 14 happen after Step 11?\n2. Why must Step 14 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 does not have to happen after Step 11, it can happen after Step 13 as well.\"\n}", "relevant_step_nos": [14, 11]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 17, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 11?", "why_question": "Why must Step 15 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 15 happen after Step 11?\n2. Why must Step 15 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 11 to ensure that the terrine is properly set and chilled before serving.\"\n}", "relevant_step_nos": [15, 11]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 18, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 11?", "why_question": "Why must Step 12 happen after Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 12 happen after Step 11?\n2. Why must Step 12 happen after Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 11 because it involves pouring the remaining cream mixture on top of the arranged vegetables.\"\n}", "relevant_step_nos": [12, 11]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 19, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 9?", "why_question": "Why must Step 15 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 15 happen after Step 9?\n2. Why must Step 15 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 9 because the terrine needs to be refrigerated for several hours until firm before serving chilled.\"\n}", "relevant_step_nos": [15, 9]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 20, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 9?", "why_question": "Why must Step 14 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 14 happen after Step 9?\n2. Why must Step 14 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 is necessary to release the terrine from the mould after it has set in Step 9.\"\n}", "relevant_step_nos": [14, 9]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 21, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 9?", "why_question": "Why must Step 13 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 13 happen after Step 9?\n2. Why must Step 13 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 9 because the terrine needs to be refrigerated for several hours until firm before it can be served chilled.\"\n}", "relevant_step_nos": [13, 9]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 22, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 1?", "why_question": "Why must Step 9 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 9 happen after Step 1?\n2. Why must Step 9 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 1 because the gelatin needs to be softened in cold water before being added to the cream mixture.\"\n}", "relevant_step_nos": [9, 1]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 23, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 1?", "why_question": "Why must Step 15 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 15 happen after Step 1?\n2. Why must Step 15 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 1 because the terrine needs to be refrigerated for several hours until firm before serving chilled.\"\n}", "relevant_step_nos": [15, 1]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 24, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 1?", "why_question": "Why must Step 8 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 8 happen after Step 1?\n2. Why must Step 8 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 1 because the gelatin needs to be softened in cold water before being added to the cream mixture.\"\n}", "relevant_step_nos": [8, 1]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 25, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 8?", "why_question": "Why must Step 15 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 15 happen after Step 8?\n2. Why must Step 15 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen after Step 8 because the terrine needs to be refrigerated for several hours until firm before it can be served chilled.\"\n}", "relevant_step_nos": [15, 8]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 26, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 14 happen after Step 12?", "why_question": "Why must Step 14 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 14 happen after Step 12?\n2. Why must Step 14 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 is necessary to release the terrine from the mould after it has set in Step 12.\"\n}", "relevant_step_nos": [14, 12]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 27, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 3?", "why_question": "Why must Step 11 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 11 happen after Step 3?\n2. Why must Step 11 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 3 because the vegetables need to be cooked and cooled before arranging them in the terrine.\"\n}", "relevant_step_nos": [11, 3]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 28, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 14?", "why_question": "Why must Step 15 happen after Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 15 happen after Step 14?\n2. Why must Step 15 happen after Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 is the final step in the procedure and can only happen after Step 14, which is the step to release the terrine from the mould.\"\n}", "relevant_step_nos": [15, 14]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 29, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 15 happen after Step 2?", "why_question": "Why must Step 15 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 15 happen after Step 2?\n2. Why must Step 15 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 15 must happen after Step 13, not Step 2, because the terrine needs to refrigerate for several hours until firm before serving chilled.\"\n}", "relevant_step_nos": [15, 2]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 0, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 10?", "why_question": "Why must Step 13 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 13 happen after Step 10?\n2. Why must Step 13 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 10 to allow the cheesecake to cool down and set properly before serving.\"\n}", "relevant_step_nos": [13, 10]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 1, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 10?", "why_question": "Why must Step 11 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 11 happen after Step 10?\n2. Why must Step 11 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The cheesecake needs to bake for 1 hour and 20 minutes before cooling down inside the oven for 1 hour.\"\n}", "relevant_step_nos": [11, 10]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 2, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 2?", "why_question": "Why must Step 11 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 11 happen after Step 2?\n2. Why must Step 11 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 2 because the cheesecake needs to be prepared and baked before it can be cooled down in the oven.\"\n}", "relevant_step_nos": [11, 2]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 3, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 2?", "why_question": "Why must Step 9 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 9 happen after Step 2?\n2. Why must Step 9 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 requires the biscuit base to be prepared in Step 2 before pouring the cheesecake mixture.\"\n}", "relevant_step_nos": [9, 2]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 4, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 2?", "why_question": "Why must Step 12 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 12 happen after Step 2?\n2. Why must Step 12 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 must happen after Step 11, not Step 2.\"\n}", "relevant_step_nos": [12, 2]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 5, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 6?", "why_question": "Why must Step 13 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 13 happen after Step 6?\n2. Why must Step 13 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cheesecake to set and chill before serving.\"\n}", "relevant_step_nos": [13, 6]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 6, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 6?", "why_question": "Why must Step 9 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 9 happen after Step 6?\n2. Why must Step 9 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 6 because the cream cheese and sugar mixture needs to be poured into the biscuit base.\"\n}", "relevant_step_nos": [9, 6]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 7, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 3?", "why_question": "Why must Step 12 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 12 happen after Step 3?\n2. Why must Step 12 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 must happen after Step 13 to allow the cheesecake to cool down and set before refrigerating.\"\n}", "relevant_step_nos": [12, 3]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 8, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 12?", "why_question": "Why must Step 13 happen after Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 13 happen after Step 12?\n2. Why must Step 13 happen after Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cheesecake to set and chill properly before serving.\"\n}", "relevant_step_nos": [13, 12]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 9, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 9?", "why_question": "Why must Step 13 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 13 happen after Step 9?\n2. Why must Step 13 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 9 to allow the cheesecake to set and chill before serving.\"\n}", "relevant_step_nos": [13, 9]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 10, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 8 because the mixture needs to be poured into the biscuit base after it has been blended and smooth.\"\n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 11, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 8?", "why_question": "Why must Step 12 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 12 happen after Step 8?\n2. Why must Step 12 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cheesecake to cool down slowly and prevent cracking.\"\n}", "relevant_step_nos": [12, 8]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 12, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 7?", "why_question": "Why must Step 9 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 9 happen after Step 7?\n2. Why must Step 9 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen after Step 7 because the mixture needs to be poured into the biscuit base after the eggs and other ingredients have been incorporated.\"\n}", "relevant_step_nos": [9, 7]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 13, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 1?", "why_question": "Why must Step 13 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 13 happen after Step 1?\n2. Why must Step 13 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 1 because the cheesecake needs to be refrigerated for at least 4 hours or overnight before serving, which can only happen after it has been baked and cooled.\"\n}", "relevant_step_nos": [13, 1]}
{"plan_idx": 55, "title": "warm-puy-lentil-and-tomato-salad", "question_idx": 0, "steps": ["Bring a large pan of water to the boil;", "add lentils and cook over low heat for 30 minutes.", "Put tomatoes in a small bowl and season with salt.", "Set aside for 15 minutes.", "Drain lentils and tomatoes and mix together.", "Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 3?", "why_question": "Why must Step 6 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: warm-puy-lentil-and-tomato-salad\n\nProcedure:\n1. Bring a large pan of water to the boil;\n2. add lentils and cook over low heat for 30 minutes.\n3. Put tomatoes in a small bowl and season with salt.\n4. Set aside for 15 minutes.\n5. Drain lentils and tomatoes and mix together.\n6. Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled.\n\n\n1. Must Step 6 happen after Step 3?\n2. Why must Step 6 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 6 must happen after Step 5, not Step 3, because the lentils and tomatoes need to be mixed together before being distributed into bowls.\"\n}", "relevant_step_nos": [6, 3]}
{"plan_idx": 55, "title": "warm-puy-lentil-and-tomato-salad", "question_idx": 1, "steps": ["Bring a large pan of water to the boil;", "add lentils and cook over low heat for 30 minutes.", "Put tomatoes in a small bowl and season with salt.", "Set aside for 15 minutes.", "Drain lentils and tomatoes and mix together.", "Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 2?", "why_question": "Why must Step 6 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: warm-puy-lentil-and-tomato-salad\n\nProcedure:\n1. Bring a large pan of water to the boil;\n2. add lentils and cook over low heat for 30 minutes.\n3. Put tomatoes in a small bowl and season with salt.\n4. Set aside for 15 minutes.\n5. Drain lentils and tomatoes and mix together.\n6. Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled.\n\n\n1. Must Step 6 happen after Step 2?\n2. Why must Step 6 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 2 because the lentils and tomatoes need to be cooked and mixed together before being distributed into individual bowls.\"\n}", "relevant_step_nos": [6, 2]}
{"plan_idx": 55, "title": "warm-puy-lentil-and-tomato-salad", "question_idx": 2, "steps": ["Bring a large pan of water to the boil;", "add lentils and cook over low heat for 30 minutes.", "Put tomatoes in a small bowl and season with salt.", "Set aside for 15 minutes.", "Drain lentils and tomatoes and mix together.", "Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 2 happen after Step 1?", "why_question": "Why must Step 2 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: warm-puy-lentil-and-tomato-salad\n\nProcedure:\n1. Bring a large pan of water to the boil;\n2. add lentils and cook over low heat for 30 minutes.\n3. Put tomatoes in a small bowl and season with salt.\n4. Set aside for 15 minutes.\n5. Drain lentils and tomatoes and mix together.\n6. Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled.\n\n\n1. Must Step 2 happen after Step 1?\n2. Why must Step 2 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 requires the water to be boiling, which is achieved in Step 1.\"\n}", "relevant_step_nos": [2, 1]}
{"plan_idx": 55, "title": "warm-puy-lentil-and-tomato-salad", "question_idx": 3, "steps": ["Bring a large pan of water to the boil;", "add lentils and cook over low heat for 30 minutes.", "Put tomatoes in a small bowl and season with salt.", "Set aside for 15 minutes.", "Drain lentils and tomatoes and mix together.", "Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 5 happen after Step 4?", "why_question": "Why must Step 5 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: warm-puy-lentil-and-tomato-salad\n\nProcedure:\n1. Bring a large pan of water to the boil;\n2. add lentils and cook over low heat for 30 minutes.\n3. Put tomatoes in a small bowl and season with salt.\n4. Set aside for 15 minutes.\n5. Drain lentils and tomatoes and mix together.\n6. Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled.\n\n\n1. Must Step 5 happen after Step 4?\n2. Why must Step 5 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen after Step 4 because the lentils and tomatoes need to be drained and mixed together before being distributed into individual bowls.\"\n}", "relevant_step_nos": [5, 4]}
{"plan_idx": 55, "title": "warm-puy-lentil-and-tomato-salad", "question_idx": 4, "steps": ["Bring a large pan of water to the boil;", "add lentils and cook over low heat for 30 minutes.", "Put tomatoes in a small bowl and season with salt.", "Set aside for 15 minutes.", "Drain lentils and tomatoes and mix together.", "Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 5?", "why_question": "Why must Step 6 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: warm-puy-lentil-and-tomato-salad\n\nProcedure:\n1. Bring a large pan of water to the boil;\n2. add lentils and cook over low heat for 30 minutes.\n3. Put tomatoes in a small bowl and season with salt.\n4. Set aside for 15 minutes.\n5. Drain lentils and tomatoes and mix together.\n6. Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled.\n\n\n1. Must Step 6 happen after Step 5?\n2. Why must Step 6 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen after Step 5 because it involves distributing the mixture into individual bowls and serving, which can only be done after the lentils and tomatoes are mixed together.\"\n}", "relevant_step_nos": [6, 5]}
{"plan_idx": 56, "title": "white bread", "question_idx": 0, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 5?", "why_question": "Why must Step 7 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 7 happen after Step 5?\n2. Why must Step 7 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 allows the dough to rise and double in bulk before moving on to the next steps.\"\n}", "relevant_step_nos": [7, 5]}
{"plan_idx": 56, "title": "white bread", "question_idx": 1, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 5?", "why_question": "Why must Step 12 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 12 happen after Step 5?\n2. Why must Step 12 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 must happen after Step 10 to allow the dough to rise and shape into loaves before baking.\"\n}", "relevant_step_nos": [12, 5]}
{"plan_idx": 56, "title": "white bread", "question_idx": 2, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 5?", "why_question": "Why must Step 11 happen after Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 11 happen after Step 5?\n2. Why must Step 11 happen after Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The dough needs to be kneaded and divided before shaping into loaves and placing in tins.\"\n}", "relevant_step_nos": [11, 5]}
{"plan_idx": 56, "title": "white bread", "question_idx": 3, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 6?", "why_question": "Why must Step 8 happen after Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 8 happen after Step 6?\n2. Why must Step 8 happen after Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the dough to rise and double in bulk.\"\n}", "relevant_step_nos": [8, 6]}
{"plan_idx": 56, "title": "white bread", "question_idx": 4, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 9?", "why_question": "Why must Step 11 happen after Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 11 happen after Step 9?\n2. Why must Step 11 happen after Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen after Step 9 because the dough needs to be divided before shaping into loaves.\"\n}", "relevant_step_nos": [11, 9]}
{"plan_idx": 56, "title": "white bread", "question_idx": 5, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 3?", "why_question": "Why must Step 10 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 10 happen after Step 3?\n2. Why must Step 10 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 does not have to happen after Step 3, as it can happen after Step 9 when the dough is ready to be shaped into loaves.\"\n}", "relevant_step_nos": [10, 3]}
{"plan_idx": 56, "title": "white bread", "question_idx": 6, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 3?", "why_question": "Why must Step 7 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 7 happen after Step 3?\n2. Why must Step 7 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 allows the dough to rise and double in bulk before moving on to the next steps.\"\n}", "relevant_step_nos": [7, 3]}
{"plan_idx": 56, "title": "white bread", "question_idx": 7, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 3?", "why_question": "Why must Step 8 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 8 happen after Step 3?\n2. Why must Step 8 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 3 because it allows the yeast to rise and the dough to double in bulk before moving on to the next step.\"\n}", "relevant_step_nos": [8, 3]}
{"plan_idx": 56, "title": "white bread", "question_idx": 8, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 11 happen after Step 3?", "why_question": "Why must Step 11 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 11 happen after Step 3?\n2. Why must Step 11 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 does not need to happen after Step 3 because the shaping of the loaves can occur after the yeast mixture has been prepared.\"\n}", "relevant_step_nos": [11, 3]}
{"plan_idx": 56, "title": "white bread", "question_idx": 9, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 3?", "why_question": "Why must Step 13 happen after Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 13 happen after Step 3?\n2. Why must Step 13 happen after Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen after Step 3 because the dough needs to rise and proof before baking.\"\n}", "relevant_step_nos": [13, 3]}
{"plan_idx": 56, "title": "white bread", "question_idx": 10, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 7?", "why_question": "Why must Step 9 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 9 happen after Step 7?\n2. Why must Step 9 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Punching the dough down after it has risen allows for the release of excess gas and redistributes the yeast, resulting in a better texture for the bread.\"\n}", "relevant_step_nos": [9, 7]}
{"plan_idx": 56, "title": "white bread", "question_idx": 11, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 7?", "why_question": "Why must Step 12 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 12 happen after Step 7?\n2. Why must Step 12 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 7 to allow the dough to rise and double in bulk before shaping into loaves.\"\n}", "relevant_step_nos": [12, 7]}
{"plan_idx": 56, "title": "white bread", "question_idx": 12, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 7?", "why_question": "Why must Step 8 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 8 happen after Step 7?\n2. Why must Step 8 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the dough to rise until doubled in bulk.\"\n}", "relevant_step_nos": [8, 7]}
{"plan_idx": 56, "title": "white bread", "question_idx": 13, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 7?", "why_question": "Why must Step 10 happen after Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 10 happen after Step 7?\n2. Why must Step 10 happen after Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 7 because the dough needs to rise before it can be punched down and kneaded.\"\n}", "relevant_step_nos": [10, 7]}
{"plan_idx": 56, "title": "white bread", "question_idx": 14, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 4 happen after Step 2?", "why_question": "Why must Step 4 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 4 happen after Step 2?\n2. Why must Step 4 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen after Step 2 because the yeast needs to proof and become active before adding the flour.\"\n}", "relevant_step_nos": [4, 2]}
{"plan_idx": 56, "title": "white bread", "question_idx": 15, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 2?", "why_question": "Why must Step 8 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 8 happen after Step 2?\n2. Why must Step 8 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen after Step 2 to allow the yeast to rise and double in bulk.\"\n}", "relevant_step_nos": [8, 2]}
{"plan_idx": 56, "title": "white bread", "question_idx": 16, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 2?", "why_question": "Why must Step 12 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 12 happen after Step 2?\n2. Why must Step 12 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 must happen after Step 8, not Step 2.\"\n}", "relevant_step_nos": [12, 2]}
{"plan_idx": 56, "title": "white bread", "question_idx": 17, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 2?", "why_question": "Why must Step 9 happen after Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 9 happen after Step 2?\n2. Why must Step 9 happen after Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the yeast has proofed properly before moving on to the next step.\"\n}", "relevant_step_nos": [9, 2]}
{"plan_idx": 56, "title": "white bread", "question_idx": 18, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 1?", "why_question": "Why must Step 7 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 7 happen after Step 1?\n2. Why must Step 7 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 does not have to happen after Step 1, as it can happen at any time during the process of making the bread.\"\n}", "relevant_step_nos": [7, 1]}
{"plan_idx": 56, "title": "white bread", "question_idx": 19, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 13 happen after Step 1?", "why_question": "Why must Step 13 happen after Step 1?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 13 happen after Step 1?\n2. Why must Step 13 happen after Step 1?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 13 must happen after Step 1 because Step 13 is the final step in the procedure, and Step 1 is the first step.\"\n}", "relevant_step_nos": [13, 1]}
{"plan_idx": 56, "title": "white bread", "question_idx": 20, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 8?", "why_question": "Why must Step 10 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 10 happen after Step 8?\n2. Why must Step 10 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 8 because punching the dough down in Step 9 releases the air and then kneading in Step 10 redistributes the yeast and allows the dough to rise again.\" \n}", "relevant_step_nos": [10, 8]}
{"plan_idx": 56, "title": "white bread", "question_idx": 21, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 8?", "why_question": "Why must Step 9 happen after Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 9 happen after Step 8?\n2. Why must Step 9 happen after Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Punching the dough down after it has doubled in bulk helps to redistribute the yeast and release some of the gas bubbles, resulting in a finer texture for the bread.\"\n}", "relevant_step_nos": [9, 8]}
{"plan_idx": 56, "title": "white bread", "question_idx": 22, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 6 happen after Step 4?", "why_question": "Why must Step 6 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 6 happen after Step 4?\n2. Why must Step 6 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the dough to rise and double in bulk.\"\n}", "relevant_step_nos": [6, 4]}
{"plan_idx": 56, "title": "white bread", "question_idx": 23, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 4?", "why_question": "Why must Step 12 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 12 happen after Step 4?\n2. Why must Step 12 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 4 because the dough needs to rise before being shaped into loaves and placed in tins.\"\n}", "relevant_step_nos": [12, 4]}
{"plan_idx": 56, "title": "white bread", "question_idx": 24, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 7 happen after Step 4?", "why_question": "Why must Step 7 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 7 happen after Step 4?\n2. Why must Step 7 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the dough to rise and double in bulk before punching it down.\"\n}", "relevant_step_nos": [7, 4]}
{"plan_idx": 56, "title": "white bread", "question_idx": 25, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 8 happen after Step 4?", "why_question": "Why must Step 8 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 8 happen after Step 4?\n2. Why must Step 8 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 allows the dough to rise and double in bulk before moving on to the next steps.\"\n}", "relevant_step_nos": [8, 4]}
{"plan_idx": 56, "title": "white bread", "question_idx": 26, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 10 happen after Step 4?", "why_question": "Why must Step 10 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 10 happen after Step 4?\n2. Why must Step 10 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen after Step 4 because the dough needs to be kneaded and allowed to rise before shaping into loaves.\"\n}", "relevant_step_nos": [10, 4]}
{"plan_idx": 56, "title": "white bread", "question_idx": 27, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 9 happen after Step 4?", "why_question": "Why must Step 9 happen after Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 9 happen after Step 4?\n2. Why must Step 9 happen after Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Punching the dough down after it has risen allows for the release of excess gas and redistributes the yeast, resulting in a better texture and flavor of the bread.\"\n}", "relevant_step_nos": [9, 4]}
{"plan_idx": 56, "title": "white bread", "question_idx": 28, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_after", "binary_question": "Must Step 12 happen after Step 10?", "why_question": "Why must Step 12 happen after Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 12 happen after Step 10?\n2. Why must Step 12 happen after Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen after Step 10 because the dough needs to rise and expand in the loaf tins before baking.\"\n}", "relevant_step_nos": [12, 10]}
