{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 0, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 8 to ensure that the ground almonds are well combined with the mixture before adding the remaining flour and milk.\"\n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 1, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 9?", "why_question": "Why must Step 6 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 6 happen before Step 9?\n2. Why must Step 6 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the ground almonds are evenly distributed throughout the cake mixture before adding the remaining flour and milk.\"\n}", "relevant_step_nos": [6, 9]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 2, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 14?", "why_question": "Why must Step 6 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 6 happen before Step 14?\n2. Why must Step 6 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 14 because the ground almonds need to be stirred into the mixture before baking, and the cake needs to be baked before serving.\"\n}", "relevant_step_nos": [6, 14]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 3, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the ground almonds need to be incorporated into the mixture before adding the flour and milk.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 4, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 11?", "why_question": "Why must Step 6 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 6 happen before Step 11?\n2. Why must Step 6 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 11 because the ground almonds need to be stirred into the mixture before it is spooned into the tin and smoothed over with the apples.\"\n}", "relevant_step_nos": [6, 11]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 5, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 13?", "why_question": "Why must Step 6 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 6 happen before Step 13?\n2. Why must Step 6 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 13 because the ground almonds need to be stirred into the mixture before baking, and the cake needs to cool slightly before serving.\"\n}", "relevant_step_nos": [6, 13]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 6, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the flour and milk are evenly combined with the mixture.\"\n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 7, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 13?", "why_question": "Why must Step 8 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 8 happen before Step 13?\n2. Why must Step 8 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 13 because the cake needs to be baked before it can cool and be turned onto a serving plate.\"\n}", "relevant_step_nos": [8, 13]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 8, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 14?", "why_question": "Why must Step 8 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 8 happen before Step 14?\n2. Why must Step 8 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 14 because the mixture needs to be prepared and baked before serving the cake.\"\n}", "relevant_step_nos": [8, 14]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 9, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 11?", "why_question": "Why must Step 8 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 8 happen before Step 11?\n2. Why must Step 8 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 11 to ensure that the flour and milk are well combined with the mixture before it is spooned into the tin.\"\n}", "relevant_step_nos": [8, 11]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 10, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 12?", "why_question": "Why must Step 8 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 8 happen before Step 12?\n2. Why must Step 8 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 12 to ensure that the flour and milk are well combined with the other ingredients before baking.\"\n}", "relevant_step_nos": [8, 12]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 11, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 12?", "why_question": "Why must Step 11 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 11 happen before Step 12?\n2. Why must Step 11 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 12 because the cake mixture needs to be spooned into the tin and smoothed before baking.\"\n}", "relevant_step_nos": [11, 12]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 12, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 14?", "why_question": "Why must Step 5 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 5 happen before Step 14?\n2. Why must Step 5 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 14 because the eggs need to be beaten well before the cake is baked.\"\n}", "relevant_step_nos": [5, 14]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 13, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 11?", "why_question": "Why must Step 5 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 5 happen before Step 11?\n2. Why must Step 5 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 11 because the eggs need to be beaten well into the mixture before spooning it into the tin and smoothing the surface.\"\n}", "relevant_step_nos": [5, 11]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 14, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the eggs need to be well beaten before adding the ground almonds.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 15, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 8 to ensure the eggs are well incorporated into the mixture before adding the flour and milk.\"\n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 16, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because the flour and milk need to be added gradually and in stages to ensure proper mixing and consistency of the batter.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 17, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 7?", "why_question": "Why must Step 4 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 4 happen before Step 7?\n2. Why must Step 4 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 7 because the eggs need to be beaten well with the butter and sugar before adding the flour and milk.\"\n}", "relevant_step_nos": [4, 7]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 18, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 13?", "why_question": "Why must Step 4 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 4 happen before Step 13?\n2. Why must Step 4 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 13 because beating the butter, sugar, and vanilla is part of the cake preparation process, while setting aside in the pan is the final step before serving.\"\n}", "relevant_step_nos": [4, 13]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 19, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 8 because beating the butter, sugar, and vanilla together creates the base for the cake mixture before adding the flour and milk.\"\n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 20, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 10?", "why_question": "Why must Step 3 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 3 happen before Step 10?\n2. Why must Step 3 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 10 to ensure that the baking parchment is in place to prevent the cake from sticking to the tin and to make it easier to remove the cake after baking.\"\n}", "relevant_step_nos": [3, 10]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 21, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 12?", "why_question": "Why must Step 3 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 3 happen before Step 12?\n2. Why must Step 3 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 12 to ensure that the cake can be easily removed from the tin after baking.\"\n}", "relevant_step_nos": [3, 12]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 22, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 13?", "why_question": "Why must Step 9 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 9 happen before Step 13?\n2. Why must Step 9 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 13 because the cake needs to be baked before it can cool and be turned onto a serving plate.\"\n}", "relevant_step_nos": [9, 13]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 23, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 8?", "why_question": "Why must Step 7 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 7 happen before Step 8?\n2. Why must Step 7 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure the flour and milk are evenly combined with the mixture.\"\n}", "relevant_step_nos": [7, 8]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 24, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 13?", "why_question": "Why must Step 7 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 7 happen before Step 13?\n2. Why must Step 7 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 13 because the cake needs to be baked before it can be set aside to cool.\"\n}", "relevant_step_nos": [7, 13]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 25, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 14?", "why_question": "Why must Step 13 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 13 happen before Step 14?\n2. Why must Step 13 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 14 to allow the cake to cool slightly and set before serving.\"\n}", "relevant_step_nos": [13, 14]}
{"plan_idx": 0, "title": "almond and apple cake", "question_idx": 26, "steps": ["Preheat the oven to 170 C / Gas mark 3.", "Brush around 22cm (base measurement) cake tin with melted butter to grease.", "Line base and side with baking parchment.", "Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).", "Add the eggs, 1 at a time, beating well after each addition.", "Stir in ground almonds.", "Add half the flour and half the milk.", "Use a wooden spoon to stir until well combined.", "Repeat remaining flour and milk.", "Arrange the apple, slightly overlapping, over the base of the prepared tin.", "Spoon mixture into the tin and smooth the surface.", "Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.", "Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 14?", "why_question": "Why must Step 2 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almond and apple cake\n\nProcedure:\n1. Preheat the oven to 170 C / Gas mark 3.\n2. Brush around 22cm (base measurement) cake tin with melted butter to grease.\n3. Line base and side with baking parchment.\n4. Beat butter, caster sugar and vanilla in a bowl for 8 mins or till pale and creamy (by hand or electric beater).\n5. Add the eggs, 1 at a time, beating well after each addition.\n6. Stir in ground almonds.\n7. Add half the flour and half the milk.\n8. Use a wooden spoon to stir until well combined.\n9. Repeat remaining flour and milk.\n10. Arrange the apple, slightly overlapping, over the base of the prepared tin.\n11. Spoon mixture into the tin and smooth the surface.\n12. Bake in the oven for 30-35 mins or until a skewer inserted into the centre comes out clean.\n13. Set aside in the pan for 10 mins to cool slightly before turning onto a serving plate.\n14. Serve warm.\n\n\n1. Must Step 2 happen before Step 14?\n2. Why must Step 2 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 14 to ensure the cake does not stick to the tin and can be easily removed for serving.\"\n}", "relevant_step_nos": [2, 14]}
{"plan_idx": 1, "title": "almost no fat banana bread", "question_idx": 0, "steps": ["Preheat oven to 180 C / Gas mark 4.", "Lightly grease a 20x10cm (8x4 in) loaf tin.", "In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.", "Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.", "Pour batter into prepared tin.", "Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.", "Turn out onto wire rack and allow to cool before slicing."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almost no fat banana bread\n\nProcedure:\n1. Preheat oven to 180 C / Gas mark 4.\n2. Lightly grease a 20x10cm (8x4 in) loaf tin.\n3. In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.\n4. Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.\n5. Pour batter into prepared tin.\n6. Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.\n7. Turn out onto wire rack and allow to cool before slicing.\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 7 because the bread needs to be baked before it can be cooled and sliced.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 1, "title": "almost no fat banana bread", "question_idx": 1, "steps": ["Preheat oven to 180 C / Gas mark 4.", "Lightly grease a 20x10cm (8x4 in) loaf tin.", "In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.", "Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.", "Pour batter into prepared tin.", "Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.", "Turn out onto wire rack and allow to cool before slicing."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almost no fat banana bread\n\nProcedure:\n1. Preheat oven to 180 C / Gas mark 4.\n2. Lightly grease a 20x10cm (8x4 in) loaf tin.\n3. In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.\n4. Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.\n5. Pour batter into prepared tin.\n6. Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.\n7. Turn out onto wire rack and allow to cool before slicing.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the bread needs to cool down after baking before it can be sliced.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 1, "title": "almost no fat banana bread", "question_idx": 2, "steps": ["Preheat oven to 180 C / Gas mark 4.", "Lightly grease a 20x10cm (8x4 in) loaf tin.", "In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.", "Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.", "Pour batter into prepared tin.", "Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.", "Turn out onto wire rack and allow to cool before slicing."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 7?", "why_question": "Why must Step 2 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almost no fat banana bread\n\nProcedure:\n1. Preheat oven to 180 C / Gas mark 4.\n2. Lightly grease a 20x10cm (8x4 in) loaf tin.\n3. In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.\n4. Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.\n5. Pour batter into prepared tin.\n6. Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.\n7. Turn out onto wire rack and allow to cool before slicing.\n\n\n1. Must Step 2 happen before Step 7?\n2. Why must Step 2 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 7 to ensure that the loaf tin is prepared and ready to receive the batter before baking.\" \n}", "relevant_step_nos": [2, 7]}
{"plan_idx": 1, "title": "almost no fat banana bread", "question_idx": 3, "steps": ["Preheat oven to 180 C / Gas mark 4.", "Lightly grease a 20x10cm (8x4 in) loaf tin.", "In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.", "Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.", "Pour batter into prepared tin.", "Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.", "Turn out onto wire rack and allow to cool before slicing."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almost no fat banana bread\n\nProcedure:\n1. Preheat oven to 180 C / Gas mark 4.\n2. Lightly grease a 20x10cm (8x4 in) loaf tin.\n3. In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.\n4. Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.\n5. Pour batter into prepared tin.\n6. Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.\n7. Turn out onto wire rack and allow to cool before slicing.\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 6 because the dry ingredients need to be mixed together before adding the wet ingredients and baking the batter.\"\n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 1, "title": "almost no fat banana bread", "question_idx": 4, "steps": ["Preheat oven to 180 C / Gas mark 4.", "Lightly grease a 20x10cm (8x4 in) loaf tin.", "In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.", "Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.", "Pour batter into prepared tin.", "Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.", "Turn out onto wire rack and allow to cool before slicing."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: almost no fat banana bread\n\nProcedure:\n1. Preheat oven to 180 C / Gas mark 4.\n2. Lightly grease a 20x10cm (8x4 in) loaf tin.\n3. In a large bowl, stir together flour, sugar, baking powder, bicarbonate of soda and cinnamon.\n4. Add egg whites, bananas and apple pur\u00c3\u00a9e; stir just until combined.\n5. Pour batter into prepared tin.\n6. Bake in preheated oven for 50 to 55 minutes, until a skewer inserted into centre of loaf comes out clean.\n7. Turn out onto wire rack and allow to cool before slicing.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because the dry ingredients need to be mixed together before adding the wet ingredients.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 2, "title": "apple-pumpkin-spread", "question_idx": 0, "steps": ["In a medium saucepan over medium heat, mix pumpkin puree, apple sauce, apple juice, dark brown soft sugar and spices.", "Bring mixture to the boil.", "Reduce heat and simmer approximately 1 1/2 hours.", "Transfer to sterile containers and chill in the refrigerator until serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: apple-pumpkin-spread\n\nProcedure:\n1. In a medium saucepan over medium heat, mix pumpkin puree, apple sauce, apple juice, dark brown soft sugar and spices.\n2. Bring mixture to the boil.\n3. Reduce heat and simmer approximately 1 1/2 hours.\n4. Transfer to sterile containers and chill in the refrigerator until serving.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because the mixture needs to be simmered before transferring to sterile containers and chilling in the refrigerator.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 2, "title": "apple-pumpkin-spread", "question_idx": 1, "steps": ["In a medium saucepan over medium heat, mix pumpkin puree, apple sauce, apple juice, dark brown soft sugar and spices.", "Bring mixture to the boil.", "Reduce heat and simmer approximately 1 1/2 hours.", "Transfer to sterile containers and chill in the refrigerator until serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: apple-pumpkin-spread\n\nProcedure:\n1. In a medium saucepan over medium heat, mix pumpkin puree, apple sauce, apple juice, dark brown soft sugar and spices.\n2. Bring mixture to the boil.\n3. Reduce heat and simmer approximately 1 1/2 hours.\n4. Transfer to sterile containers and chill in the refrigerator until serving.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 because the mixture needs to be brought to a boil before being transferred to sterile containers and chilled.\" \n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 0, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 8 because the chicken needs to be arranged on the serving platter before adding the mozzarella, basil, and tomato.\"\n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 1, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 because the chicken needs to be saut\u00e9ed in the frying pan after being coated with oil.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 2, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 7 because the chicken needs to be cooked before adding the mozzarella, basil, and tomato.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 3, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is marinating the chicken, which needs to happen before pouring the vinaigrette over it in Step 2.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 4, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 8?", "why_question": "Why must Step 1 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 1 happen before Step 8?\n2. Why must Step 1 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The chicken needs to be marinated before it can be cooked and served with the mozzarella, basil, and tomato.\"\n}", "relevant_step_nos": [1, 8]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 5, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 10?", "why_question": "Why must Step 1 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 1 happen before Step 10?\n2. Why must Step 1 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 10 because the chicken needs to marinate for 12 to 24 hours before it can be cooked and served.\"\n}", "relevant_step_nos": [1, 10]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 6, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the chicken needs to be cooked before it is arranged on the serving platter.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 7, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 8 because the chicken needs to be cooked before adding the mozzarella, basil, and tomato.\" \n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 8, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because the chicken needs to be cooked before it can be served with the balsamic vinegar and other toppings.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 9, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 7?", "why_question": "Why must Step 4 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 4 happen before Step 7?\n2. Why must Step 4 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 7 because the chicken needs to be cooked before adding the mozzarella, basil, and tomato.\"\n}", "relevant_step_nos": [4, 7]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 10, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 10?", "why_question": "Why must Step 8 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 8 happen before Step 10?\n2. Why must Step 8 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 does not need to happen before Step 10, as the balsamic vinegar can be drizzled over the platter after serving the chicken.\"\n}", "relevant_step_nos": [8, 10]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 11, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 7?", "why_question": "Why must Step 2 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 2 happen before Step 7?\n2. Why must Step 2 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 7 because marinating the chicken in the vinaigrette adds flavor to the chicken, which is essential before adding the mozzarella, basil, and tomato in Step 7.\"\n}", "relevant_step_nos": [2, 7]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 12, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 because the chicken needs to marinate in the vinaigrette for 12 to 24 hours before it is removed and cooked in Step 4.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 13, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 10?", "why_question": "Why must Step 9 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 9 happen before Step 10?\n2. Why must Step 9 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 10 because the balsamic vinegar needs to be drizzled over the platter before seasoning with salt and pepper.\" \n}", "relevant_step_nos": [9, 10]}
{"plan_idx": 3, "title": "balsamic-and-mozzarella-chicken", "question_idx": 14, "steps": ["Place chicken in a shallow dish or large resealable plastic bag.", "Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.", "Coat a large frying pan lightly with oil, and set over low heat.", "Remove the chicken from the marinade and discard the marinade.", "Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.", "Arrange chicken on a serving platter.", "Place a generous slice of fresh mozzarella on top of each piece.", "Place a leaf of basil on top of the cheese, and cover with a slice of tomato.", "Drizzle balsamic vinegar over the platter, then season with salt and pepper.", "Serve!"], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 10?", "why_question": "Why must Step 7 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: balsamic-and-mozzarella-chicken\n\nProcedure:\n1. Place chicken in a shallow dish or large resealable plastic bag.\n2. Pour the vinaigrette over it, cover or seal and marinate in the fridge for 12 to 24 hours.\n3. Coat a large frying pan lightly with oil, and set over low heat.\n4. Remove the chicken from the marinade and discard the marinade.\n5. Saute chicken breast fillets over low heat for about 30 minutes, or until juices run clear.\n6. Arrange chicken on a serving platter.\n7. Place a generous slice of fresh mozzarella on top of each piece.\n8. Place a leaf of basil on top of the cheese, and cover with a slice of tomato.\n9. Drizzle balsamic vinegar over the platter, then season with salt and pepper.\n10. Serve!\n\n\n1. Must Step 7 happen before Step 10?\n2. Why must Step 7 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 does not need to happen before Step 10, as the balsamic vinegar can be drizzled over the platter after serving the chicken.\"\n}", "relevant_step_nos": [7, 10]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 0, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 because the mixture needs to be pressed into the tin before baking.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 1, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 11?", "why_question": "Why must Step 2 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 2 happen before Step 11?\n2. Why must Step 2 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 11 because the base needs to be prepared and baked before adding the other ingredients and toppings.\"\n}", "relevant_step_nos": [2, 11]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 2, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 9?", "why_question": "Why must Step 2 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 2 happen before Step 9?\n2. Why must Step 2 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 9 because the chopped pecans and margarine need to be combined with the flour and pressed into the tin before the bananas and pineapple are arranged on top.\"\n}", "relevant_step_nos": [2, 9]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 3, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because the base needs to be baked and cooled before adding the filling.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 4, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 9 because the filling needs to be spread over the cooled base before arranging the bananas on top.\"\n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 5, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 11?", "why_question": "Why must Step 8 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 8 happen before Step 11?\n2. Why must Step 8 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 11 because the whipped cream is spread over the filling, so the filling needs to be in place first.\"\n}", "relevant_step_nos": [8, 11]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 6, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 10?", "why_question": "Why must Step 8 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 8 happen before Step 10?\n2. Why must Step 8 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 10 because the whipped cream needs to be the final layer on top of the strawberries.\" \n}", "relevant_step_nos": [8, 10]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 7, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 4?", "why_question": "Why must Step 1 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 1 happen before Step 4?\n2. Why must Step 1 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 preheats the oven before baking the base in Step 4.\"\n}", "relevant_step_nos": [1, 4]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 8, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 8?", "why_question": "Why must Step 1 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 1 happen before Step 8?\n2. Why must Step 1 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 8 because the base needs to be baked and cooled before the filling is spread over it.\"\n}", "relevant_step_nos": [1, 8]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 9, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 11?", "why_question": "Why must Step 1 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 1 happen before Step 11?\n2. Why must Step 1 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 11 because the oven needs to be preheated before baking the cake.\"\n}", "relevant_step_nos": [1, 11]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 10, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 10?", "why_question": "Why must Step 4 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 4 happen before Step 10?\n2. Why must Step 4 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 10 because the base needs to be cooled before adding the bananas and other toppings.\"\n}", "relevant_step_nos": [4, 10]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 11, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 11?", "why_question": "Why must Step 4 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 4 happen before Step 11?\n2. Why must Step 4 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 11 because the base needs to be baked and cooled before adding the whipped cream and other toppings.\"\n}", "relevant_step_nos": [4, 11]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 12, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 10?", "why_question": "Why must Step 5 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 5 happen before Step 10?\n2. Why must Step 5 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 10 because the base needs to cool before adding the bananas and strawberries.\"\n}", "relevant_step_nos": [5, 10]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 13, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 9?", "why_question": "Why must Step 7 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 7 happen before Step 9?\n2. Why must Step 7 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 9 because the filling needs to be spread over the cooled base before arranging the bananas on top.\"\n}", "relevant_step_nos": [7, 9]}
{"plan_idx": 4, "title": "banana-split-cake", "question_idx": 14, "steps": ["Preheat oven to 190 C / Gas 5.", "In a large bowl, combine flour, 175g margarine and chopped pecans.", "Press into the bottom of a 20x30cm tin.", "Bake in preheated oven for 20 minutes.", "Remove from oven and allow to cool.", "In a large bowl, combine remaining margarine, eggs and icing sugar.", "Beat with an electric mixer for 6 minutes or until fluffy.", "Spread over cooled base.", "Slice bananas and arrange on filling, then top with pineapple.", "Slice strawberries and layer over bananas.", "Cover with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: banana-split-cake\n\nProcedure:\n1. Preheat oven to 190 C / Gas 5.\n2. In a large bowl, combine flour, 175g margarine and chopped pecans.\n3. Press into the bottom of a 20x30cm tin.\n4. Bake in preheated oven for 20 minutes.\n5. Remove from oven and allow to cool.\n6. In a large bowl, combine remaining margarine, eggs and icing sugar.\n7. Beat with an electric mixer for 6 minutes or until fluffy.\n8. Spread over cooled base.\n9. Slice bananas and arrange on filling, then top with pineapple.\n10. Slice strawberries and layer over bananas.\n11. Cover with whipped cream.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the ingredients need to be combined before beating with an electric mixer.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 5, "title": "best-ever-margarita", "question_idx": 0, "steps": ["Rim 2 margarita glasses with salt if desired and fill with ice.", "Pour the tequila, lime cordial, triple sec and lemon-lime soft drink into a cocktail shaker filled with ice.", "Hold your hand firmly over over the top of the shaker so the top doesn't pop off from the fizzy soft drink and shake vigorously.", "Pour into prepared margarita glasses and serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: best-ever-margarita\n\nProcedure:\n1. Rim 2 margarita glasses with salt if desired and fill with ice.\n2. Pour the tequila, lime cordial, triple sec and lemon-lime soft drink into a cocktail shaker filled with ice.\n3. Hold your hand firmly over over the top of the shaker so the top doesn't pop off from the fizzy soft drink and shake vigorously.\n4. Pour into prepared margarita glasses and serve.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 because the ingredients need to be mixed in the cocktail shaker before pouring into the glasses.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 6, "title": "black-rice-pudding", "question_idx": 0, "steps": ["Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.", "Bring to the boil over high heat, stirring frequently.", "Reduce heat to medium-low.", "Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.", "Stir in coconut cream and vanilla extract.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: black-rice-pudding\n\nProcedure:\n1. Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.\n2. Bring to the boil over high heat, stirring frequently.\n3. Reduce heat to medium-low.\n4. Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.\n5. Stir in coconut cream and vanilla extract.\n6. Serve warm.\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to prepare the rice before boiling it in Step 2.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 6, "title": "black-rice-pudding", "question_idx": 1, "steps": ["Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.", "Bring to the boil over high heat, stirring frequently.", "Reduce heat to medium-low.", "Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.", "Stir in coconut cream and vanilla extract.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 5?", "why_question": "Why must Step 1 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: black-rice-pudding\n\nProcedure:\n1. Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.\n2. Bring to the boil over high heat, stirring frequently.\n3. Reduce heat to medium-low.\n4. Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.\n5. Stir in coconut cream and vanilla extract.\n6. Serve warm.\n\n\n1. Must Step 1 happen before Step 5?\n2. Why must Step 1 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 5 because the rice needs to be cooked and tender before adding the coconut cream and vanilla extract.\"\n}", "relevant_step_nos": [1, 5]}
{"plan_idx": 6, "title": "black-rice-pudding", "question_idx": 2, "steps": ["Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.", "Bring to the boil over high heat, stirring frequently.", "Reduce heat to medium-low.", "Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.", "Stir in coconut cream and vanilla extract.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: black-rice-pudding\n\nProcedure:\n1. Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.\n2. Bring to the boil over high heat, stirring frequently.\n3. Reduce heat to medium-low.\n4. Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.\n5. Stir in coconut cream and vanilla extract.\n6. Serve warm.\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to prepare the rice before cooking it in Step 3.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 6, "title": "black-rice-pudding", "question_idx": 3, "steps": ["Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.", "Bring to the boil over high heat, stirring frequently.", "Reduce heat to medium-low.", "Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.", "Stir in coconut cream and vanilla extract.", "Serve warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: black-rice-pudding\n\nProcedure:\n1. Rinse and drain glutinous black rice and glutinous white rice, then combine with sugar and water in a large saucepan.\n2. Bring to the boil over high heat, stirring frequently.\n3. Reduce heat to medium-low.\n4. Cover and cook until rice is tender and creamy, stirring occasionally, about 45 minutes.\n5. Stir in coconut cream and vanilla extract.\n6. Serve warm.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because the rice needs to be cooked and tender before adding the coconut cream and vanilla extract.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 0, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 10?", "why_question": "Why must Step 4 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 4 happen before Step 10?\n2. Why must Step 4 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 10 because the dough needs to be mixed well before baking.\"\n}", "relevant_step_nos": [4, 10]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 1, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 7?", "why_question": "Why must Step 4 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 4 happen before Step 7?\n2. Why must Step 4 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 7 because the dough needs to be mixed well before rolling it out to the desired thickness.\"\n}", "relevant_step_nos": [4, 7]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 2, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because the dough needs to be mixed well before it is rolled out and placed on the baking trays.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 3, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because creaming the butter and brown sugar is the first step in preparing the dough, and kneading the dough in Step 6 requires the creamed mixture.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 4, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 9?", "why_question": "Why must Step 2 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 2 happen before Step 9?\n2. Why must Step 2 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 9 because creaming the butter and brown sugar is part of the initial mixing process before the shortbread is shaped and baked.\"\n}", "relevant_step_nos": [2, 9]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 5, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 10?", "why_question": "Why must Step 8 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 8 happen before Step 10?\n2. Why must Step 8 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 10 because the shortbread needs to be cut into strips before baking.\" \n}", "relevant_step_nos": [8, 10]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 6, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 10?", "why_question": "Why must Step 6 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 6 happen before Step 10?\n2. Why must Step 6 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 10 because the dough needs to be kneaded and prepared before baking.\" \n}", "relevant_step_nos": [6, 10]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 7, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Kneading the dough before rolling it out helps to evenly distribute the flour and create a smoother texture.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 8, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 8 because the dough needs to be kneaded and the right consistency before it can be rolled and cut into strips.\"\n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 9, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 9?", "why_question": "Why must Step 3 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 3 happen before Step 9?\n2. Why must Step 3 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 9 because the dough needs to be prepared before it can be placed on the baking trays.\"\n}", "relevant_step_nos": [3, 9]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 10, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because the flour needs to be added to the creamed butter and brown sugar in order to make the dough.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 11, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 8?", "why_question": "Why must Step 3 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 3 happen before Step 8?\n2. Why must Step 3 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 8 because the dough needs to be prepared before it can be rolled and cut into strips.\"\n}", "relevant_step_nos": [3, 8]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 12, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 6 because the flour needs to be added to the butter and sugar mixture before kneading to form the dough.\"\n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 13, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 10?", "why_question": "Why must Step 3 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 3 happen before Step 10?\n2. Why must Step 3 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 10 because the dough needs to be prepared before baking.\" \n}", "relevant_step_nos": [3, 10]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 14, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 10?", "why_question": "Why must Step 5 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 5 happen before Step 10?\n2. Why must Step 5 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 10 to prepare the surface for kneading and rolling the dough.\"\n}", "relevant_step_nos": [5, 10]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 15, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 to ensure that the dough does not stick to the board while kneading.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 7, "title": "brown sugar shortbread", "question_idx": 16, "steps": ["Preheat oven to 160 C / Gas mark 3.", "Cream butter and brown sugar.", "Add 400g to 500g flour.", "Mix well.", "Sprinkle board with the remaining flour.", "Knead for 5 minutes, adding enough flour to make a soft dough.", "Roll to 1cm (1/2 in) thickness.", "Cut into 7x2cm (3x1 in) strips.", "Prick with fork and place on ungreased baking trays.", "Bake in preheated oven for 20 to 25 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 10?", "why_question": "Why must Step 9 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: brown sugar shortbread\n\nProcedure:\n1. Preheat oven to 160 C / Gas mark 3.\n2. Cream butter and brown sugar.\n3. Add 400g to 500g flour.\n4. Mix well.\n5. Sprinkle board with the remaining flour.\n6. Knead for 5 minutes, adding enough flour to make a soft dough.\n7. Roll to 1cm (1/2 in) thickness.\n8. Cut into 7x2cm (3x1 in) strips.\n9. Prick with fork and place on ungreased baking trays.\n10. Bake in preheated oven for 20 to 25 minutes.\n\n\n1. Must Step 9 happen before Step 10?\n2. Why must Step 9 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 10 to prepare the shortbread for baking by pricking it with a fork and placing it on the baking trays.\"\n}", "relevant_step_nos": [9, 10]}
{"plan_idx": 8, "title": "busters-liver-dinner", "question_idx": 0, "steps": ["Put all ingredients into a slow cooker, mix together.", "Put slow cooker on high for 4 to 5 hours or low all day.", "( If mixture starts to look too dry just add a little hot water.", ")", "Once cooked, divide between several plastic microwaveable tubs with lids.", "This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.", "When ready to eat microwave with lid half off for 2 to 3 minutes on medium high."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: busters-liver-dinner\n\nProcedure:\n1. Put all ingredients into a slow cooker, mix together.\n2. Put slow cooker on high for 4 to 5 hours or low all day.\n3. ( If mixture starts to look too dry just add a little hot water.\n4. )\n5. Once cooked, divide between several plastic microwaveable tubs with lids.\n6. This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.\n7. When ready to eat microwave with lid half off for 2 to 3 minutes on medium high.\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 3 because the ingredients need to be mixed together before adding hot water if necessary.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 8, "title": "busters-liver-dinner", "question_idx": 1, "steps": ["Put all ingredients into a slow cooker, mix together.", "Put slow cooker on high for 4 to 5 hours or low all day.", "( If mixture starts to look too dry just add a little hot water.", ")", "Once cooked, divide between several plastic microwaveable tubs with lids.", "This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.", "When ready to eat microwave with lid half off for 2 to 3 minutes on medium high."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: busters-liver-dinner\n\nProcedure:\n1. Put all ingredients into a slow cooker, mix together.\n2. Put slow cooker on high for 4 to 5 hours or low all day.\n3. ( If mixture starts to look too dry just add a little hot water.\n4. )\n5. Once cooked, divide between several plastic microwaveable tubs with lids.\n6. This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.\n7. When ready to eat microwave with lid half off for 2 to 3 minutes on medium high.\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 7 because the mixture needs to be divided into tubs before it can be microwaved for individual meals.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 8, "title": "busters-liver-dinner", "question_idx": 2, "steps": ["Put all ingredients into a slow cooker, mix together.", "Put slow cooker on high for 4 to 5 hours or low all day.", "( If mixture starts to look too dry just add a little hot water.", ")", "Once cooked, divide between several plastic microwaveable tubs with lids.", "This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.", "When ready to eat microwave with lid half off for 2 to 3 minutes on medium high."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: busters-liver-dinner\n\nProcedure:\n1. Put all ingredients into a slow cooker, mix together.\n2. Put slow cooker on high for 4 to 5 hours or low all day.\n3. ( If mixture starts to look too dry just add a little hot water.\n4. )\n5. Once cooked, divide between several plastic microwaveable tubs with lids.\n6. This mixture usually makes 6 to 7 meals, I freeze 4 and put the rest in the fridge.\n7. When ready to eat microwave with lid half off for 2 to 3 minutes on medium high.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because the ingredients need to be cooked before they can be divided into separate meals.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 9, "title": "butternut-squash-pastry-for-pies", "question_idx": 0, "steps": ["Place the flour, egg and squash into the container of a food processor.", "Pulse until mixed into a doughy ball.", "You may need to add a little extra flour.", "Divide into two equal parts.", "Use in a pie or pastie recipe."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: butternut-squash-pastry-for-pies\n\nProcedure:\n1. Place the flour, egg and squash into the container of a food processor.\n2. Pulse until mixed into a doughy ball.\n3. You may need to add a little extra flour.\n4. Divide into two equal parts.\n5. Use in a pie or pastie recipe.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 to ensure the dough has the right consistency for use in a pie or pastie recipe.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 9, "title": "butternut-squash-pastry-for-pies", "question_idx": 1, "steps": ["Place the flour, egg and squash into the container of a food processor.", "Pulse until mixed into a doughy ball.", "You may need to add a little extra flour.", "Divide into two equal parts.", "Use in a pie or pastie recipe."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: butternut-squash-pastry-for-pies\n\nProcedure:\n1. Place the flour, egg and squash into the container of a food processor.\n2. Pulse until mixed into a doughy ball.\n3. You may need to add a little extra flour.\n4. Divide into two equal parts.\n5. Use in a pie or pastie recipe.\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 2 because the ingredients need to be combined before pulsing them into a doughy ball.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 0, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 16?", "why_question": "Why must Step 8 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 8 happen before Step 16?\n2. Why must Step 8 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The liquid mixture needs to thicken before baking in the oven.\"\n}", "relevant_step_nos": [8, 16]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 1, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 13?", "why_question": "Why must Step 8 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 8 happen before Step 13?\n2. Why must Step 8 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The liquid mixture needs to be thickened before it can be poured over the chicken mixture and covered with pastry.\"\n}", "relevant_step_nos": [8, 13]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 2, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 15?", "why_question": "Why must Step 8 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 8 happen before Step 15?\n2. Why must Step 8 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 15 because the liquid mixture needs to be thickened before pouring it over the chicken mixture and baking the pie.\"\n}", "relevant_step_nos": [8, 15]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 3, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 12?", "why_question": "Why must Step 9 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 9 happen before Step 12?\n2. Why must Step 9 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 12 because the hot liquid mixture needs to be poured over the chicken mixture before covering with the top pastry.\"\n}", "relevant_step_nos": [9, 12]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 4, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 15?", "why_question": "Why must Step 9 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 9 happen before Step 15?\n2. Why must Step 9 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 15 because the hot liquid mixture needs to be prepared before pouring it over the chicken mixture and covering with pastry.\" \n}", "relevant_step_nos": [9, 15]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 5, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 13?", "why_question": "Why must Step 9 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 9 happen before Step 13?\n2. Why must Step 9 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 13 because the hot liquid mixture needs to be poured over the chicken mixture before covering with the top pastry.\" \n}", "relevant_step_nos": [9, 13]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 6, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 15?", "why_question": "Why must Step 1 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 1 happen before Step 15?\n2. Why must Step 1 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 15 because the oven needs to be preheated before baking the pie.\"\n}", "relevant_step_nos": [1, 15]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 7, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 15?", "why_question": "Why must Step 2 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 2 happen before Step 15?\n2. Why must Step 2 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The chicken and vegetable mixture needs to be prepared before it can be baked in the oven.\"\n}", "relevant_step_nos": [2, 15]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 8, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 14?", "why_question": "Why must Step 2 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 2 happen before Step 14?\n2. Why must Step 2 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The chicken mixture needs to be prepared and placed in the pie dish before baking in the oven.\"\n}", "relevant_step_nos": [2, 14]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 9, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 14?", "why_question": "Why must Step 4 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 4 happen before Step 14?\n2. Why must Step 4 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The chicken mixture needs to be prepared and set aside before it can be placed in the pie dish and covered with pastry.\"\n}", "relevant_step_nos": [4, 14]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 10, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 16?", "why_question": "Why must Step 4 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 4 happen before Step 16?\n2. Why must Step 4 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The chicken mixture needs to be drained and set aside before it can be placed in the pie dish and baked in the oven.\"\n}", "relevant_step_nos": [4, 16]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 11, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 12?", "why_question": "Why must Step 4 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 4 happen before Step 12?\n2. Why must Step 4 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The chicken mixture needs to be drained and set aside before it can be placed in the pie dish in Step 12.\"\n}", "relevant_step_nos": [4, 12]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 12, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 14?", "why_question": "Why must Step 7 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 7 happen before Step 14?\n2. Why must Step 7 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 14 because the hot liquid mixture needs to be poured over the chicken mixture before the pie is covered and baked.\" \n}", "relevant_step_nos": [7, 14]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 13, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 9?", "why_question": "Why must Step 7 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 7 happen before Step 9?\n2. Why must Step 7 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 9 because the liquid mixture needs to be thickened before being removed from heat and set aside.\"\n}", "relevant_step_nos": [7, 9]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 14, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 8?", "why_question": "Why must Step 7 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 7 happen before Step 8?\n2. Why must Step 7 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 8 because the chicken stock and milk need to be slowly stirred in to create a thick mixture.\"\n}", "relevant_step_nos": [7, 8]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 15, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 12?", "why_question": "Why must Step 7 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 7 happen before Step 12?\n2. Why must Step 7 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 12 because the hot liquid mixture needs to be poured over the chicken mixture before covering with the top pastry.\"\n}", "relevant_step_nos": [7, 12]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 16, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 8 because the onion needs to be cooked in butter before adding the flour, salt, pepper, and celery seed to create the base for the sauce.\" \n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 17, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 14?", "why_question": "Why must Step 5 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 5 happen before Step 14?\n2. Why must Step 5 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 14 because the pastry needs to be prepared and placed on top of the pie before baking.\" \n}", "relevant_step_nos": [5, 14]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 18, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 11?", "why_question": "Why must Step 10 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 10 happen before Step 11?\n2. Why must Step 10 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 11 because the pie dish needs to be prepared with pastry lining before placing the chicken mixture in it.\"\n}", "relevant_step_nos": [10, 11]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 19, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 15?", "why_question": "Why must Step 10 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 10 happen before Step 15?\n2. Why must Step 10 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 15 because the pie dish needs to be prepared and lined with pastry before baking.\" \n}", "relevant_step_nos": [10, 15]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 20, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 14 happen before Step 16?", "why_question": "Why must Step 14 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 14 happen before Step 16?\n2. Why must Step 14 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen before Step 16 because the pie needs to cool for 10 minutes after baking to allow the filling to set before serving.\"\n}", "relevant_step_nos": [14, 16]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 21, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 16?", "why_question": "Why must Step 12 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 12 happen before Step 16?\n2. Why must Step 12 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen before Step 16 because the pie needs to be baked before it can be cooled and served.\"\n}", "relevant_step_nos": [12, 16]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 22, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 16?", "why_question": "Why must Step 6 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 6 happen before Step 16?\n2. Why must Step 6 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 16 because the filling needs to be prepared and thickened before it is added to the pie and baked.\" \n}", "relevant_step_nos": [6, 16]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 23, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 14?", "why_question": "Why must Step 13 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 13 happen before Step 14?\n2. Why must Step 13 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 14 to allow steam to escape during baking.\"\n}", "relevant_step_nos": [13, 14]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 24, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 16?", "why_question": "Why must Step 13 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 13 happen before Step 16?\n2. Why must Step 13 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 16 to ensure that the pie has the proper crust and is fully cooked before serving.\"\n}", "relevant_step_nos": [13, 16]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 25, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 13?", "why_question": "Why must Step 11 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 11 happen before Step 13?\n2. Why must Step 11 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 13 to ensure that the chicken mixture is properly covered and sealed with the top pastry.\" \n}", "relevant_step_nos": [11, 13]}
{"plan_idx": 10, "title": "chicken and vegetable pie", "question_idx": 26, "steps": ["Preheat oven to 220 C / Gas mark 7.", "In a saucepan, combine chicken, carrots, peas and celery.", "Add water to cover and boil for 15 minutes.", "Remove from heat, drain and set aside.", "In the saucepan over medium heat, cook onion in butter until soft and translucent.", "Stir in flour, salt, pepper and celery seed.", "Slowly stir in chicken stock and milk.", "Simmer over medium-low heat until thick.", "Remove from heat and set aside.", "Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.", "Place the chicken mixture in bottom of the prepared pie dish.", "Pour hot liquid mixture over.", "Cover with top pastry, seal edges, and cut away excess pastry.", "Make several small slits in the top to allow steam to escape.", "Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.", "Cool for 10 minutes before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 14?", "why_question": "Why must Step 3 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken and vegetable pie\n\nProcedure:\n1. Preheat oven to 220 C / Gas mark 7.\n2. In a saucepan, combine chicken, carrots, peas and celery.\n3. Add water to cover and boil for 15 minutes.\n4. Remove from heat, drain and set aside.\n5. In the saucepan over medium heat, cook onion in butter until soft and translucent.\n6. Stir in flour, salt, pepper and celery seed.\n7. Slowly stir in chicken stock and milk.\n8. Simmer over medium-low heat until thick.\n9. Remove from heat and set aside.\n10. Prepare a 23cm (9 in) pie dish by lining with pastry, reserving pastry to cover.\n11. Place the chicken mixture in bottom of the prepared pie dish.\n12. Pour hot liquid mixture over.\n13. Cover with top pastry, seal edges, and cut away excess pastry.\n14. Make several small slits in the top to allow steam to escape.\n15. Bake in the preheated oven for 30 to 35 minutes, or until pastry is golden brown and filling is bubbly.\n16. Cool for 10 minutes before serving.\n\n\n1. Must Step 3 happen before Step 14?\n2. Why must Step 3 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 14 because the chicken mixture needs to be boiled and prepared before being placed in the pie dish and baked.\"\n}", "relevant_step_nos": [3, 14]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 0, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 8?", "why_question": "Why must Step 1 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 1 happen before Step 8?\n2. Why must Step 1 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 8 because the mixture of prawns, chicken, egg, and cornflour needs to be cooked before adding salt and sherry in Step 7.\"\n}", "relevant_step_nos": [1, 8]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 1, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to prepare the prawns and chicken before cooking them in the hot oil in Step 3.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 2, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 6?", "why_question": "Why must Step 1 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 1 happen before Step 6?\n2. Why must Step 1 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 6 because the prawns and chicken mixture needs to be added to the pot with the stock before bringing it to a boil.\"\n}", "relevant_step_nos": [1, 6]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 3, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 4?", "why_question": "Why must Step 1 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 1 happen before Step 4?\n2. Why must Step 1 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The prawns and chicken need to be mixed with the egg and cornflour before they are cooked in the oil.\"\n}", "relevant_step_nos": [1, 4]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 4, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 9?", "why_question": "Why must Step 1 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 1 happen before Step 9?\n2. Why must Step 1 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 9 because the prawns and chicken mixture needs to be cooked before being added to the soup to ensure it is safe to eat.\"\n}", "relevant_step_nos": [1, 9]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 5, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 7?", "why_question": "Why must Step 1 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 1 happen before Step 7?\n2. Why must Step 1 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 7 because the prawns and chicken mixture needs to be prepared before adding salt and sherry to the soup.\"\n}", "relevant_step_nos": [1, 7]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 6, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 12?", "why_question": "Why must Step 3 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 3 happen before Step 12?\n2. Why must Step 3 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The prawns and chicken mixture needs to be cooked before adding it to the soup in Step 12.\"\n}", "relevant_step_nos": [3, 12]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 7, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 7 because the prawns and chicken mixture needs to be cooked before adding salt and sherry to the soup.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 8, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 8?", "why_question": "Why must Step 3 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 3 happen before Step 8?\n2. Why must Step 3 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 8 because the prawns and chicken mixture needs to be cooked before adding salt and sherry and returning to the boil.\"\n}", "relevant_step_nos": [3, 8]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 9, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because the prawns and chicken mixture needs to be cooked in the hot oil before draining.\" \n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 10, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 13?", "why_question": "Why must Step 3 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 3 happen before Step 13?\n2. Why must Step 3 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 13 because the prawns and chicken mixture needs to be cooked before being added to the soup.\"\n}", "relevant_step_nos": [3, 13]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 11, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The prawns and chicken mixture needs to be cooked before being added to the pot with the stock and other ingredients.\"\n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 12, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to heat the oil before adding the prawns and chicken mixture in Step 3, which is then added to the pot in Step 5 before bringing to the boil in Step 6.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 13, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 13?", "why_question": "Why must Step 2 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 2 happen before Step 13?\n2. Why must Step 2 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 13 because the oil needs to be heated before adding the rice for browning.\"\n}", "relevant_step_nos": [2, 13]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 14, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 8?", "why_question": "Why must Step 2 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 2 happen before Step 8?\n2. Why must Step 2 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 8 because the oil needs to be hot before adding the prawns and chicken mixture in Step 3, and then the remaining oil needs to be hot for browning the rice in Step 11.\"\n}", "relevant_step_nos": [2, 8]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 15, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 is necessary to cook the prawns and chicken before adding salt and sherry in Step 7.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 16, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 12?", "why_question": "Why must Step 5 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 5 happen before Step 12?\n2. Why must Step 5 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 12 because the mixture needs to be cooked with the other ingredients before adding the rice to the soup.\"\n}", "relevant_step_nos": [5, 12]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 17, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 8 because the mixture needs to be cooked before adding salt and sherry and returning to the boil.\"\n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 18, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 13?", "why_question": "Why must Step 7 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 7 happen before Step 13?\n2. Why must Step 7 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 adds flavor to the soup and must happen before serving.\"\n}", "relevant_step_nos": [7, 13]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 19, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 13?", "why_question": "Why must Step 8 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 8 happen before Step 13?\n2. Why must Step 8 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 13 because the soup needs to be brought to a boil again after adding salt and sherry before serving.\"\n}", "relevant_step_nos": [8, 13]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 20, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 13?", "why_question": "Why must Step 11 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 11 happen before Step 13?\n2. Why must Step 11 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The rice needs to be browned quickly before being added to the soup in order to achieve the desired texture and flavor.\"\n}", "relevant_step_nos": [11, 13]}
{"plan_idx": 11, "title": "chicken-prawn-and-rice-soup", "question_idx": 21, "steps": ["Mix together the prawns, chicken, egg and cornflour.", "Heat 750ml of the oil in wok or deep frying pan.", "When it is hot, add prawns and chicken mixture.", "Cook for 1/2 minute and drain.", "Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.", "Bring to the boil.", "Add salt and sherry.", "Return to the boil.", "Reduce heat and allow to simmer.", "Meanwhile, heat the remaining 250ml oil until it is hot.", "Add rice and brown quickly.", "Drain and add to soup.", "Serve and enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicken-prawn-and-rice-soup\n\nProcedure:\n1. Mix together the prawns, chicken, egg and cornflour.\n2. Heat 750ml of the oil in wok or deep frying pan.\n3. When it is hot, add prawns and chicken mixture.\n4. Cook for 1/2 minute and drain.\n5. Place above mixture in pot with the stock, mushrooms, water chestnuts, bamboo shoots and green beans.\n6. Bring to the boil.\n7. Add salt and sherry.\n8. Return to the boil.\n9. Reduce heat and allow to simmer.\n10. Meanwhile, heat the remaining 250ml oil until it is hot.\n11. Add rice and brown quickly.\n12. Drain and add to soup.\n13. Serve and enjoy!\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the ingredients need to be brought to a boil before adding salt and sherry.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 12, "title": "chicory-and-roquefort-salad", "question_idx": 0, "steps": ["In a bowl, toss chicory, Roquefort and pecan nuts together.", "Make the dressing by mixing together cream cheese, yoghurt and salt and pepper.", "Drizzle over salad."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chicory-and-roquefort-salad\n\nProcedure:\n1. In a bowl, toss chicory, Roquefort and pecan nuts together.\n2. Make the dressing by mixing together cream cheese, yoghurt and salt and pepper.\n3. Drizzle over salad.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 3 because the dressing needs to be made before it can be drizzled over the salad.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 0, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 21?", "why_question": "Why must Step 1 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 1 happen before Step 21?\n2. Why must Step 1 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 21 because the cake needs to be preheated before it can be baked, and the cooling process cannot begin until the cake is baked.\"\n}", "relevant_step_nos": [1, 21]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 1, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 17?", "why_question": "Why must Step 1 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 1 happen before Step 17?\n2. Why must Step 1 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 17 because the oven needs to be preheated before the cake can be baked.\"\n}", "relevant_step_nos": [1, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 2, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 22?", "why_question": "Why must Step 10 happen before Step 22?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 10 happen before Step 22?\n2. Why must Step 10 happen before Step 22?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 does not need to happen before Step 22, as Step 22 is for decoration after the cake has cooled completely.\"\n}", "relevant_step_nos": [10, 22]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 3, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 16?", "why_question": "Why must Step 10 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 10 happen before Step 16?\n2. Why must Step 10 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 16 because the chocolate mixture needs to be added to the cake tin before baking.\"\n}", "relevant_step_nos": [10, 16]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 4, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 18?", "why_question": "Why must Step 10 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 10 happen before Step 18?\n2. Why must Step 10 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 involves adding the cooled chocolate to the mixture, which is necessary before baking in Step 18.\"\n}", "relevant_step_nos": [10, 18]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 5, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 21?", "why_question": "Why must Step 10 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 10 happen before Step 21?\n2. Why must Step 10 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 21 because the cake needs to cool completely before adding icing sugar for decoration.\"\n}", "relevant_step_nos": [10, 21]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 6, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 17?", "why_question": "Why must Step 10 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 10 happen before Step 17?\n2. Why must Step 10 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 17 because the mixture needs to be poured into the cake tin before baking.\"\n}", "relevant_step_nos": [10, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 7, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 16?", "why_question": "Why must Step 12 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 12 happen before Step 16?\n2. Why must Step 12 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen before Step 16 because the egg white mixture needs to be folded into the chocolate mixture before pouring the mixture into the cake tin.\"\n}", "relevant_step_nos": [12, 16]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 8, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 17?", "why_question": "Why must Step 12 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 12 happen before Step 17?\n2. Why must Step 12 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen before Step 17 because the egg whites need to be beaten and folded into the chocolate mixture before pouring the mixture into the cake tin and baking.\"\n}", "relevant_step_nos": [12, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 9, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 21?", "why_question": "Why must Step 12 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 12 happen before Step 21?\n2. Why must Step 12 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 12 does not need to happen before Step 21 as they are independent of each other.\"\n}", "relevant_step_nos": [12, 21]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 10, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 18?", "why_question": "Why must Step 12 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 12 happen before Step 18?\n2. Why must Step 12 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen before Step 18 because the cake needs to be baked immediately after the batter is prepared and poured into the tin.\"\n}", "relevant_step_nos": [12, 18]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 11, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 18?", "why_question": "Why must Step 4 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 4 happen before Step 18?\n2. Why must Step 4 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 18 because the mixture needs to be smooth and cooled before baking the cake.\"\n}", "relevant_step_nos": [4, 18]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 12, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 22?", "why_question": "Why must Step 4 happen before Step 22?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 4 happen before Step 22?\n2. Why must Step 4 happen before Step 22?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 22 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [4, 22]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 13, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 17?", "why_question": "Why must Step 4 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 4 happen before Step 17?\n2. Why must Step 4 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 17 because the chocolate mixture needs to be prepared and set aside to cool before being added to the cake mixture and baked.\"\n}", "relevant_step_nos": [4, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 14, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 13?", "why_question": "Why must Step 4 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 4 happen before Step 13?\n2. Why must Step 4 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 13 because the chocolate mixture needs to be prepared and cooled before folding it into the egg white mixture.\"\n}", "relevant_step_nos": [4, 13]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 15, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 21?", "why_question": "Why must Step 4 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 4 happen before Step 21?\n2. Why must Step 4 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 21 because the cake needs to cool and set before removing it from the tin and allowing it to cool completely.\"\n}", "relevant_step_nos": [4, 21]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 16, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 17?", "why_question": "Why must Step 9 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 9 happen before Step 17?\n2. Why must Step 9 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 17 because the mixture needs to be poured into the cake tin before baking.\"\n}", "relevant_step_nos": [9, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 17, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 16?", "why_question": "Why must Step 9 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 9 happen before Step 16?\n2. Why must Step 9 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 16 because the mixture needs to be prepared before pouring it into the cake tin.\"\n}", "relevant_step_nos": [9, 16]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 18, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 21?", "why_question": "Why must Step 9 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 9 happen before Step 21?\n2. Why must Step 9 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 21 because the cake needs to be allowed to cool completely after baking, and Step 9 is part of the baking process.\"\n}", "relevant_step_nos": [9, 21]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 19, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 21?", "why_question": "Why must Step 3 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 3 happen before Step 21?\n2. Why must Step 3 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 21 because the chocolate mixture needs to cool before the cake can be removed from the tin and allowed to cool completely.\"\n}", "relevant_step_nos": [3, 21]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 20, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 because the chocolate mixture needs to cool before being added to the other ingredients.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 21, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 10?", "why_question": "Why must Step 3 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 3 happen before Step 10?\n2. Why must Step 3 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 10 because the chocolate needs to be cooled before being added to the mixture.\"\n}", "relevant_step_nos": [3, 10]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 22, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 22?", "why_question": "Why must Step 3 happen before Step 22?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 3 happen before Step 22?\n2. Why must Step 3 happen before Step 22?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 22 because the chocolate mixture needs to cool before the icing sugar can be sifted over the top for decoration.\"\n}", "relevant_step_nos": [3, 22]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 23, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 8 because beating the butter and margarine together is a prerequisite for gradually beating in the sugar until light and fluffy.\"\n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 24, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 13?", "why_question": "Why must Step 6 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 6 happen before Step 13?\n2. Why must Step 6 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 13 because the egg whites need to be beaten separately and then folded into the chocolate mixture.\" \n}", "relevant_step_nos": [6, 13]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 25, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 16?", "why_question": "Why must Step 6 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 6 happen before Step 16?\n2. Why must Step 6 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 16 because the mixture needs to be prepared before pouring it into the cake tin.\"\n}", "relevant_step_nos": [6, 16]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 26, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 18?", "why_question": "Why must Step 6 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 6 happen before Step 18?\n2. Why must Step 6 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 18 is the baking process, so it must happen before Step 6 which involves mixing the butter and margarine.\"\n}", "relevant_step_nos": [6, 18]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 27, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 17?", "why_question": "Why must Step 6 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 6 happen before Step 17?\n2. Why must Step 6 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 17 because the mixture needs to be prepared and poured into the cake tin before baking.\"\n}", "relevant_step_nos": [6, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 28, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 21?", "why_question": "Why must Step 6 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 6 happen before Step 21?\n2. Why must Step 6 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 21 because the cake needs to cool in the tin for 10 minutes to set before being removed and allowed to cool completely.\" \n}", "relevant_step_nos": [6, 21]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 29, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 9?", "why_question": "Why must Step 6 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 6 happen before Step 9?\n2. Why must Step 6 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 9 because the butter and margarine need to be beaten together before gradually adding the flour.\"\n}", "relevant_step_nos": [6, 9]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 30, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 10?", "why_question": "Why must Step 6 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 6 happen before Step 10?\n2. Why must Step 6 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 10 because the cooled chocolate needs to be added to the butter and margarine mixture before the flour is added.\"\n}", "relevant_step_nos": [6, 10]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 31, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 17?", "why_question": "Why must Step 13 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 13 happen before Step 17?\n2. Why must Step 13 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 17 to ensure that the egg white mixture is properly folded into the chocolate mixture before baking.\"\n}", "relevant_step_nos": [13, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 32, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 22?", "why_question": "Why must Step 13 happen before Step 22?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 13 happen before Step 22?\n2. Why must Step 13 happen before Step 22?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 22 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [13, 22]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 33, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 21?", "why_question": "Why must Step 13 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 13 happen before Step 21?\n2. Why must Step 13 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 21 because the cake needs to set and cool before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [13, 21]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 34, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 22?", "why_question": "Why must Step 5 happen before Step 22?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 5 happen before Step 22?\n2. Why must Step 5 happen before Step 22?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 22 because the cake needs to cool completely before sifting icing sugar over the top for decoration.\"\n}", "relevant_step_nos": [5, 22]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 35, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 13?", "why_question": "Why must Step 5 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 5 happen before Step 13?\n2. Why must Step 5 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The chocolate mixture needs to cool before being folded into the egg white mixture.\"\n}", "relevant_step_nos": [5, 13]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 36, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 18?", "why_question": "Why must Step 5 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 5 happen before Step 18?\n2. Why must Step 5 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 18 because the mixture needs to cool before baking the cake.\"\n}", "relevant_step_nos": [5, 18]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 37, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 17?", "why_question": "Why must Step 5 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 5 happen before Step 17?\n2. Why must Step 5 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The chocolate mixture needs to cool before being added to the cake tin and baked.\"\n}", "relevant_step_nos": [5, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 38, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 20?", "why_question": "Why must Step 5 happen before Step 20?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 5 happen before Step 20?\n2. Why must Step 5 happen before Step 20?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 20 because allowing the cake to set for 10 minutes in the tin helps it hold its shape and prevents it from falling apart when removed.\"\n}", "relevant_step_nos": [5, 20]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 39, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 10?", "why_question": "Why must Step 7 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 7 happen before Step 10?\n2. Why must Step 7 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 10 because the cooled chocolate needs to be gradually beaten into the butter, margarine, and sugar mixture.\"\n}", "relevant_step_nos": [7, 10]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 40, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 21?", "why_question": "Why must Step 7 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 7 happen before Step 21?\n2. Why must Step 7 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 21 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [7, 21]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 41, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 17?", "why_question": "Why must Step 7 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 7 happen before Step 17?\n2. Why must Step 7 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 17 because the mixture needs to be prepared and ready to pour into the cake tin before baking.\"\n}", "relevant_step_nos": [7, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 42, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 16?", "why_question": "Why must Step 7 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 7 happen before Step 16?\n2. Why must Step 7 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 16 because the mixture needs to be prepared before pouring it into the cake tin.\"\n}", "relevant_step_nos": [7, 16]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 43, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 10?", "why_question": "Why must Step 8 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 8 happen before Step 10?\n2. Why must Step 8 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 10 because the egg yolks need to be gradually beaten in before adding the cooled chocolate to the mix.\"\n}", "relevant_step_nos": [8, 10]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 44, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 22?", "why_question": "Why must Step 8 happen before Step 22?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 8 happen before Step 22?\n2. Why must Step 8 happen before Step 22?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 22 because the egg yolks need to be gradually beaten in with the sugar to create a light and fluffy mixture before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [8, 22]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 45, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 16?", "why_question": "Why must Step 8 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 8 happen before Step 16?\n2. Why must Step 8 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 16 because the mixture needs to be prepared before pouring it into the cake tin.\"\n}", "relevant_step_nos": [8, 16]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 46, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 17?", "why_question": "Why must Step 8 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 8 happen before Step 17?\n2. Why must Step 8 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 17 because the mixture needs to be prepared and ready to pour into the cake tin before baking.\"\n}", "relevant_step_nos": [8, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 47, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 13?", "why_question": "Why must Step 8 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 8 happen before Step 13?\n2. Why must Step 8 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 13 to ensure that the egg yolks are well incorporated into the mixture before adding the egg white mixture.\"\n}", "relevant_step_nos": [8, 13]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 48, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 15 happen before Step 17?", "why_question": "Why must Step 15 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 15 happen before Step 17?\n2. Why must Step 15 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen before Step 17 to prepare the cake tin for baking and prevent the cake from sticking to the tin.\"\n}", "relevant_step_nos": [15, 17]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 49, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 15 happen before Step 21?", "why_question": "Why must Step 15 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 15 happen before Step 21?\n2. Why must Step 15 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 15 must happen before Step 21 because the cake needs to cool and set in the tin before being removed and allowed to cool completely.\"\n}", "relevant_step_nos": [15, 21]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 50, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 10?", "why_question": "Why must Step 2 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 2 happen before Step 10?\n2. Why must Step 2 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 10 because the cooled chocolate needs to be added to the mixture in Step 10.\"\n}", "relevant_step_nos": [2, 10]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 51, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The chocolate mixture needs to cool before being added to the butter and sugar mixture.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 52, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 13?", "why_question": "Why must Step 2 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 2 happen before Step 13?\n2. Why must Step 2 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 13 because the chocolate mixture needs to be prepared before folding it into the egg white mixture.\"\n}", "relevant_step_nos": [2, 13]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 53, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 17 happen before Step 22?", "why_question": "Why must Step 17 happen before Step 22?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 17 happen before Step 22?\n2. Why must Step 17 happen before Step 22?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 17 must happen before Step 22 because the cake needs to be baked and cooled before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [17, 22]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 54, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 16 happen before Step 22?", "why_question": "Why must Step 16 happen before Step 22?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 16 happen before Step 22?\n2. Why must Step 16 happen before Step 22?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen before Step 22 because the cake needs to cool completely before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [16, 22]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 55, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 16?", "why_question": "Why must Step 11 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 11 happen before Step 16?\n2. Why must Step 11 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 16 because the egg white mixture needs to be folded into the chocolate mixture before pouring the mixture into the cake tin.\"\n}", "relevant_step_nos": [11, 16]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 56, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 18 happen before Step 22?", "why_question": "Why must Step 18 happen before Step 22?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 18 happen before Step 22?\n2. Why must Step 18 happen before Step 22?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 18 must happen before Step 22 because the cake needs to be fully baked and cooled before adding the icing sugar for decoration.\"\n}", "relevant_step_nos": [18, 22]}
{"plan_idx": 13, "title": "chocolate-chocolate-cake", "question_idx": 57, "steps": ["Preheat the oven to 180 degrees C / Gas Mark 4.", "Place the chocolate chips, coffee and water in a microwave safe bowl.", "Microwave until chocolate is almost melted.", "Remove and whisk until mixture is smooth.", "Set aside to cool.", "Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.", "Gradually beat in sugar until light and fluffy.", "Gradually beat in egg yolks one at a time, beating well after each one.", "Slowly add the flour and beat at a low speed.", "Add the cooled chocolate to the mix and beat again.", "In a separate bowl, beat egg whites until foamy.", "Then increase speed and beat until stiff.", "Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.", "Grease a 23cm springform tin or a cake tin with deep sides.", "Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.", "Pour the mixture into the cake tin.", "Bake for I hour.", "The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.", "Note that the cake is very dense, so the top may crack when baking.", "Cool in the tin for 10 minutes to allow the cake to set.", "Then remove from the tin, and allow to cool completely, which may take several hours.", "Sift icing sugar lightly over the top for decoration."], "question_type": "dependent_real_before", "binary_question": "Must Step 14 happen before Step 22?", "why_question": "Why must Step 14 happen before Step 22?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-chocolate-cake\n\nProcedure:\n1. Preheat the oven to 180 degrees C / Gas Mark 4.\n2. Place the chocolate chips, coffee and water in a microwave safe bowl.\n3. Microwave until chocolate is almost melted.\n4. Remove and whisk until mixture is smooth.\n5. Set aside to cool.\n6. Using a hand held mixer, beat butter and margarine together in a large bowl until creamy.\n7. Gradually beat in sugar until light and fluffy.\n8. Gradually beat in egg yolks one at a time, beating well after each one.\n9. Slowly add the flour and beat at a low speed.\n10. Add the cooled chocolate to the mix and beat again.\n11. In a separate bowl, beat egg whites until foamy.\n12. Then increase speed and beat until stiff.\n13. Fold egg white mixture, one third at a time, into the chocolate mixture until blended thoroughly.\n14. Grease a 23cm springform tin or a cake tin with deep sides.\n15. Wrap the bottom of the tin securely with greaseproof paper or foil, so it comes up about 1 .5cm on the sides.\n16. Pour the mixture into the cake tin.\n17. Bake for I hour.\n18. The cake is ready when a skewer inserted about 1 .5cm from the side of the tin comes out clean.\n19. Note that the cake is very dense, so the top may crack when baking.\n20. Cool in the tin for 10 minutes to allow the cake to set.\n21. Then remove from the tin, and allow to cool completely, which may take several hours.\n22. Sift icing sugar lightly over the top for decoration.\n\n\n1. Must Step 14 happen before Step 22?\n2. Why must Step 14 happen before Step 22?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen before Step 22 because the cake needs to be baked before it can be decorated with icing sugar.\"\n}", "relevant_step_nos": [14, 22]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 0, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 11?", "why_question": "Why must Step 10 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 10 happen before Step 11?\n2. Why must Step 10 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 11 to ensure that the pancakes are kept warm as they are being cooked.\" \n}", "relevant_step_nos": [10, 11]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 1, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 to ensure that all the ingredients are properly mixed together before allowing the batter to rest.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 2, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 10?", "why_question": "Why must Step 3 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 3 happen before Step 10?\n2. Why must Step 3 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 10 because Step 3 involves mixing the pancake batter, which is necessary before cooking the pancakes in Step 10.\"\n}", "relevant_step_nos": [3, 10]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 3, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 9?", "why_question": "Why must Step 3 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 3 happen before Step 9?\n2. Why must Step 3 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 9 because the batter needs to be prepared before it can be used to cook the pancakes.\"\n}", "relevant_step_nos": [3, 9]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 4, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 11?", "why_question": "Why must Step 3 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 3 happen before Step 11?\n2. Why must Step 3 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 11 because Step 3 involves preparing the pancake batter, which is necessary before cooking the pancakes in Step 11.\"\n}", "relevant_step_nos": [3, 11]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 5, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 8?", "why_question": "Why must Step 3 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 3 happen before Step 8?\n2. Why must Step 3 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 8 because the batter needs to be prepared before it can be cooked.\"\n}", "relevant_step_nos": [3, 8]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 6, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 9?", "why_question": "Why must Step 5 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 5 happen before Step 9?\n2. Why must Step 5 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 9 because the frying pan needs to be heated before cooking the pancake and flipping it.\"\n}", "relevant_step_nos": [5, 9]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 7, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the frying pan needs to be heated before greasing it with oil.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 8, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 7 because the frying pan needs to be heated before pouring the batter to cook the pancakes.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 9, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 11?", "why_question": "Why must Step 9 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 9 happen before Step 11?\n2. Why must Step 9 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 11 because the pancakes need to be cooked on both sides before stacking to keep warm.\"\n}", "relevant_step_nos": [9, 11]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 10, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 11?", "why_question": "Why must Step 1 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 1 happen before Step 11?\n2. Why must Step 1 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 11 because the pancake batter needs to be prepared before cooking the pancakes.\"\n}", "relevant_step_nos": [1, 11]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 11, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 4?", "why_question": "Why must Step 1 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 1 happen before Step 4?\n2. Why must Step 1 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Sifting the flour and cocoa powder before resting the batter allows for better incorporation of the dry ingredients and smoother batter.\"\n}", "relevant_step_nos": [1, 4]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 12, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 10?", "why_question": "Why must Step 1 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 1 happen before Step 10?\n2. Why must Step 1 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 10 because the batter needs to be prepared before cooking the pancakes.\"\n}", "relevant_step_nos": [1, 10]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 13, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 10?", "why_question": "Why must Step 2 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 2 happen before Step 10?\n2. Why must Step 2 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 10 because the batter needs to be prepared before cooking the pancakes.\"\n}", "relevant_step_nos": [2, 10]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 14, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 9?", "why_question": "Why must Step 2 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 2 happen before Step 9?\n2. Why must Step 2 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 9 because the beaten eggs need to be whisked into the batter before cooking the pancakes.\"\n}", "relevant_step_nos": [2, 9]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 15, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 7?", "why_question": "Why must Step 2 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 2 happen before Step 7?\n2. Why must Step 2 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 7 because the eggs need to be whisked into the flour and cocoa powder before adding the other ingredients to create a smooth batter.\"\n}", "relevant_step_nos": [2, 7]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 16, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 3 because the beaten eggs need to be incorporated into the dry ingredients before adding the liquid ingredients.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 17, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the batter to rest and develop flavor before cooking.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 18, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 11?", "why_question": "Why must Step 2 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 2 happen before Step 11?\n2. Why must Step 2 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 2 does not need to happen before Step 11 because Step 11 is the final step in the procedure and does not depend on the order of the previous steps.\"\n}", "relevant_step_nos": [2, 11]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 19, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 11?", "why_question": "Why must Step 7 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 7 happen before Step 11?\n2. Why must Step 7 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 11 because the pancakes need to be cooked before they can be stacked to keep warm.\"\n}", "relevant_step_nos": [7, 11]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 20, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 allows the pancake batter to rest and develop flavor before cooking, which is important for the final texture and taste of the pancakes.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 21, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 allows the batter to rest and develop flavor before cooking the pancakes.\"\n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 22, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 9?", "why_question": "Why must Step 6 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 6 happen before Step 9?\n2. Why must Step 6 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 9 to ensure the pancake does not stick to the pan while cooking.\"\n}", "relevant_step_nos": [6, 9]}
{"plan_idx": 14, "title": "chocolate-pancake-batter", "question_idx": 23, "steps": ["Sift the flour and cocoa powder into a bowl.", "Make a well in the centre and whisk in the beaten eggs.", "Gradually whisk in milk, melted butter and sugar.", "For best results, set the pancake batter aside for 20 minutes to rest.", "Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.", "Use a piece of kitchen paper to grease the pan with a little sunflower oil.", "Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.", "Cook for a minute or so until the pancake edges start to crisp up.", "Use a palette knife to flip the pancake and cook the other side, about 1 minute.", "Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.", "Stack the pancakes on top of each other to keep warm."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: chocolate-pancake-batter\n\nProcedure:\n1. Sift the flour and cocoa powder into a bowl.\n2. Make a well in the centre and whisk in the beaten eggs.\n3. Gradually whisk in milk, melted butter and sugar.\n4. For best results, set the pancake batter aside for 20 minutes to rest.\n5. Heat a heavy-based non-stick frying pan (about 20-22cm in diameter) over moderate heat.\n6. Use a piece of kitchen paper to grease the pan with a little sunflower oil.\n7. Use a soup ladle to pour some batter into the middle of the pan, just enough to create a slightly thicker pancake than a crepe.\n8. Cook for a minute or so until the pancake edges start to crisp up.\n9. Use a palette knife to flip the pancake and cook the other side, about 1 minute.\n10. Turn out of the pan and repeat with remaining batter, greasing the pan between pancakes if necessary.\n11. Stack the pancakes on top of each other to keep warm.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 9 to ensure that the pancake edges are properly cooked before flipping.\"\n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 15, "title": "cloud-bread", "question_idx": 0, "steps": ["Preheat oven to 150 C / Gas mark 2.", "Line a baking tray with baking paper.", "Mix cream cheese and egg yolks together until smooth.", "Place egg whites and baking powder in a glass, metal, or ceramic bowl;", "beat together until stiff peaks form.", "Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.", "Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.", "Bake in the preheated oven until golden brown, 17 to 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cloud-bread\n\nProcedure:\n1. Preheat oven to 150 C / Gas mark 2.\n2. Line a baking tray with baking paper.\n3. Mix cream cheese and egg yolks together until smooth.\n4. Place egg whites and baking powder in a glass, metal, or ceramic bowl;\n5. beat together until stiff peaks form.\n6. Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.\n7. Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.\n8. Bake in the preheated oven until golden brown, 17 to 20 minutes.\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 8 because the batter needs to be prepared and folded before baking.\" \n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 15, "title": "cloud-bread", "question_idx": 1, "steps": ["Preheat oven to 150 C / Gas mark 2.", "Line a baking tray with baking paper.", "Mix cream cheese and egg yolks together until smooth.", "Place egg whites and baking powder in a glass, metal, or ceramic bowl;", "beat together until stiff peaks form.", "Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.", "Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.", "Bake in the preheated oven until golden brown, 17 to 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cloud-bread\n\nProcedure:\n1. Preheat oven to 150 C / Gas mark 2.\n2. Line a baking tray with baking paper.\n3. Mix cream cheese and egg yolks together until smooth.\n4. Place egg whites and baking powder in a glass, metal, or ceramic bowl;\n5. beat together until stiff peaks form.\n6. Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.\n7. Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.\n8. Bake in the preheated oven until golden brown, 17 to 20 minutes.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 8 because the egg whites need to be beaten to stiff peaks before being folded into the cream cheese and egg yolk mixture.\"\n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 15, "title": "cloud-bread", "question_idx": 2, "steps": ["Preheat oven to 150 C / Gas mark 2.", "Line a baking tray with baking paper.", "Mix cream cheese and egg yolks together until smooth.", "Place egg whites and baking powder in a glass, metal, or ceramic bowl;", "beat together until stiff peaks form.", "Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.", "Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.", "Bake in the preheated oven until golden brown, 17 to 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 6?", "why_question": "Why must Step 4 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cloud-bread\n\nProcedure:\n1. Preheat oven to 150 C / Gas mark 2.\n2. Line a baking tray with baking paper.\n3. Mix cream cheese and egg yolks together until smooth.\n4. Place egg whites and baking powder in a glass, metal, or ceramic bowl;\n5. beat together until stiff peaks form.\n6. Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.\n7. Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.\n8. Bake in the preheated oven until golden brown, 17 to 20 minutes.\n\n\n1. Must Step 4 happen before Step 6?\n2. Why must Step 4 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 6 because the egg whites need to be beaten to stiff peaks before being folded into the cream cheese and egg yolk mixture.\"\n}", "relevant_step_nos": [4, 6]}
{"plan_idx": 15, "title": "cloud-bread", "question_idx": 3, "steps": ["Preheat oven to 150 C / Gas mark 2.", "Line a baking tray with baking paper.", "Mix cream cheese and egg yolks together until smooth.", "Place egg whites and baking powder in a glass, metal, or ceramic bowl;", "beat together until stiff peaks form.", "Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.", "Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.", "Bake in the preheated oven until golden brown, 17 to 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cloud-bread\n\nProcedure:\n1. Preheat oven to 150 C / Gas mark 2.\n2. Line a baking tray with baking paper.\n3. Mix cream cheese and egg yolks together until smooth.\n4. Place egg whites and baking powder in a glass, metal, or ceramic bowl;\n5. beat together until stiff peaks form.\n6. Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.\n7. Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.\n8. Bake in the preheated oven until golden brown, 17 to 20 minutes.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the egg whites and baking powder need to be beaten together until stiff peaks form before folding in the cream cheese and egg yolk mixture.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 15, "title": "cloud-bread", "question_idx": 4, "steps": ["Preheat oven to 150 C / Gas mark 2.", "Line a baking tray with baking paper.", "Mix cream cheese and egg yolks together until smooth.", "Place egg whites and baking powder in a glass, metal, or ceramic bowl;", "beat together until stiff peaks form.", "Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.", "Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.", "Bake in the preheated oven until golden brown, 17 to 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cloud-bread\n\nProcedure:\n1. Preheat oven to 150 C / Gas mark 2.\n2. Line a baking tray with baking paper.\n3. Mix cream cheese and egg yolks together until smooth.\n4. Place egg whites and baking powder in a glass, metal, or ceramic bowl;\n5. beat together until stiff peaks form.\n6. Fold cream cheese and egg yolk mixture gently into egg whites until just combined and light yellow in colour.\n7. Spoon the batter onto the prepared baking tray in mounds about 10cm in diameter, flattening peaks with the back of the spoon.\n8. Bake in the preheated oven until golden brown, 17 to 20 minutes.\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 7 because the egg whites need to be beaten until stiff peaks form before being folded into the cream cheese and egg yolk mixture in Step 6.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 0, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 8 because the whipped cream with oats needs to be prepared before spooning into serving bowls and topping with fresh berries.\"\n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 1, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the whipped cream needs to be prepared before folding in the other ingredients.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 2, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 10?", "why_question": "Why must Step 5 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 5 happen before Step 10?\n2. Why must Step 5 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 10 to allow the toasted oats to cool before adding the whisky and garnish.\"\n}", "relevant_step_nos": [5, 10]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 3, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 4?", "why_question": "Why must Step 1 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 1 happen before Step 4?\n2. Why must Step 1 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 4 because the oats need to be toasted in the oven before they can be set aside to cool.\"\n}", "relevant_step_nos": [1, 4]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 4, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because the oats need to be toasted and cooled before being added to the whipped cream.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 5, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 10?", "why_question": "Why must Step 2 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 2 happen before Step 10?\n2. Why must Step 2 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 10 because the oats need to be toasted before being used as a garnish in the final step.\"\n}", "relevant_step_nos": [2, 10]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 6, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 7?", "why_question": "Why must Step 2 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 2 happen before Step 7?\n2. Why must Step 2 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Toasted oats need to cool before being folded into the whipped cream.\"\n}", "relevant_step_nos": [2, 7]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 7, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 8?", "why_question": "Why must Step 2 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 2 happen before Step 8?\n2. Why must Step 2 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Toasted oats need to be folded into the whipped cream before spooning into serving bowls.\"\n}", "relevant_step_nos": [2, 8]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 8, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Toasting the oats requires spreading them out on a baking tray first.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 9, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To prevent the cream from melting when mixed with the warm oats.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 10, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The toasted oats need to cool before being folded into the whipped cream.\"\n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 16, "title": "cranachan", "question_idx": 11, "steps": ["Preheat the oven to 180 C / Gas mark 4.", "Spread oats out in a thin layer on a baking tray.", "Toast in the oven for about 10 minutes, or until lightly browned.", "Set aside to cool.", "Remove them from the tray for faster cooling.", "In a medium bowl, whip the cream to firm peaks.", "Gently fold in the icing sugar, vanilla and toasted oats.", "Spoon into 4 serving bowls, and top with fresh berries.", "For an extra touch, drizzle some whisky over each serving.", "Garnish with a mint leaf."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 10?", "why_question": "Why must Step 3 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: cranachan\n\nProcedure:\n1. Preheat the oven to 180 C / Gas mark 4.\n2. Spread oats out in a thin layer on a baking tray.\n3. Toast in the oven for about 10 minutes, or until lightly browned.\n4. Set aside to cool.\n5. Remove them from the tray for faster cooling.\n6. In a medium bowl, whip the cream to firm peaks.\n7. Gently fold in the icing sugar, vanilla and toasted oats.\n8. Spoon into 4 serving bowls, and top with fresh berries.\n9. For an extra touch, drizzle some whisky over each serving.\n10. Garnish with a mint leaf.\n\n\n1. Must Step 3 happen before Step 10?\n2. Why must Step 3 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 10 because the oats need to be toasted before being used as a garnish in the final step.\"\n}", "relevant_step_nos": [3, 10]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 0, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 5?", "why_question": "Why must Step 1 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 1 happen before Step 5?\n2. Why must Step 1 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 5 because the stock needs to be dissolved in hot water before it can be added to the mixture in Step 5.\"\n}", "relevant_step_nos": [1, 5]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 1, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 9?", "why_question": "Why must Step 1 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 1 happen before Step 9?\n2. Why must Step 1 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 9 because the stock needs to be dissolved before it can be added to the soup mixture and pureed.\"\n}", "relevant_step_nos": [1, 9]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 2, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the stock needs to be stirred in before covering and simmering the mixture.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 3, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 9?", "why_question": "Why must Step 5 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 5 happen before Step 9?\n2. Why must Step 5 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 9 because the soup needs to be simmered and cooked before it is pureed and poured into bowls for serving.\"\n}", "relevant_step_nos": [5, 9]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 4, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 7 because the stock needs to be stirred in before covering and simmering the soup for 20 minutes.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 5, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because the squash needs to be cooked and softened before being pureed into a smooth soup.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 6, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the stock needs to be added to the squash and other ingredients before covering and simmering for 20 minutes.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 7, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 8?", "why_question": "Why must Step 2 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 2 happen before Step 8?\n2. Why must Step 2 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 8 because the butter needs to be melted in order to cook the onion, garlic, and crushed chillies before pureeing the mixture in the liquidiser.\"\n}", "relevant_step_nos": [2, 8]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 8, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 3 because melting the butter is necessary before cooking and stirring the onion, garlic, and crushed chillies in it.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 9, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 7?", "why_question": "Why must Step 2 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 2 happen before Step 7?\n2. Why must Step 2 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 7 because the butter needs to be melted before the soup is simmered for 20 minutes.\"\n}", "relevant_step_nos": [2, 7]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 10, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 9?", "why_question": "Why must Step 6 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 6 happen before Step 9?\n2. Why must Step 6 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 9 because the soup needs to simmer for 20 minutes before being pureed and served.\"\n}", "relevant_step_nos": [6, 9]}
{"plan_idx": 17, "title": "creamy-chayote-squash-soup", "question_idx": 11, "steps": ["Dissolve the stock in the hot water.", "Melt the butter in a large saucepan over medium heat.", "Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.", "Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.", "Stir in the stock and 1 tablespoon coriander;", "cover.", "Simmer about 20 minutes.", "Pour the mixture into a liquidiser and puree until smooth.", "Pour into bowls and garnish with a sprig of coriander to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: creamy-chayote-squash-soup\n\nProcedure:\n1. Dissolve the stock in the hot water.\n2. Melt the butter in a large saucepan over medium heat.\n3. Cook and stir the onion, garlic and crushed chillies in the butter until the onion is soft.\n4. Add the squash, 2 tablespoons coriander, salt and pepper and stir continuously for 5 minutes.\n5. Stir in the stock and 1 tablespoon coriander;\n6. cover.\n7. Simmer about 20 minutes.\n8. Pour the mixture into a liquidiser and puree until smooth.\n9. Pour into bowls and garnish with a sprig of coriander to serve.\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 7 because the onion and garlic need to be cooked before the soup is simmered for 20 minutes.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 0, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 9 because wrapping the foil tightly around the portions is necessary to cook the mussels in the packets on the barbecue in Step 9.\"\n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 1, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 10?", "why_question": "Why must Step 5 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 5 happen before Step 10?\n2. Why must Step 5 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 10 because the curry mixture needs to be added to the mussels before they are cooked, and any mussels that do not open after cooking should be discarded.\"\n}", "relevant_step_nos": [5, 10]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 2, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the curry mixture needs to be dotted on the mussels before sprinkling with red pepper and parsley.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 3, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 8 because the curry mixture needs to be added to the mussels before they are wrapped in foil and cooked on the barbecue.\"\n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 4, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 10?", "why_question": "Why must Step 2 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 2 happen before Step 10?\n2. Why must Step 2 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 10 because the curry mixture needs to be applied to the mussels before they are cooked, and any mussels that do not open need to be discarded after cooking.\" \n}", "relevant_step_nos": [2, 10]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 5, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because the curry mixture needs to be prepared before it can be dotted on the mussels in Step 5.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 6, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 8?", "why_question": "Why must Step 7 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 7 happen before Step 8?\n2. Why must Step 7 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 8 because the lime slices need to be placed on top of the mussels before wrapping them in foil.\"\n}", "relevant_step_nos": [7, 8]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 7, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 10?", "why_question": "Why must Step 1 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 1 happen before Step 10?\n2. Why must Step 1 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 10 because it is necessary to preheat the barbecue before cooking the mussels, and Step 10 involves discarding any mussels that do not open during cooking.\"\n}", "relevant_step_nos": [1, 10]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 8, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 9?", "why_question": "Why must Step 1 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 1 happen before Step 9?\n2. Why must Step 1 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 9 because the barbecue needs to be preheated before cooking the mussels.\"\n}", "relevant_step_nos": [1, 9]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 9, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 10?", "why_question": "Why must Step 6 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 6 happen before Step 10?\n2. Why must Step 6 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 10 because the mussels need to be cooked with the curry mixture and other ingredients before checking for any that do not open.\"\n}", "relevant_step_nos": [6, 10]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 10, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 9?", "why_question": "Why must Step 6 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 6 happen before Step 9?\n2. Why must Step 6 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 9 because the curry mixture, red pepper, and parsley need to be added to the mussels before they are cooked in the foil packets.\"\n}", "relevant_step_nos": [6, 9]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 11, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 8 because the mussels need to be divided into portions and placed on foil before wrapping them tightly and cooking them on the barbecue.\"\n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 12, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 10?", "why_question": "Why must Step 4 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 4 happen before Step 10?\n2. Why must Step 4 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 10 because the mussels need to be divided into portions and wrapped in foil before cooking, and any mussels that do not open need to be discarded after cooking.\" \n}", "relevant_step_nos": [4, 10]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 13, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 10?", "why_question": "Why must Step 3 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 3 happen before Step 10?\n2. Why must Step 3 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 10 because the mussels need to be divided into portions and wrapped in foil before cooking and checking for any unopened mussels.\"\n}", "relevant_step_nos": [3, 10]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 14, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 8?", "why_question": "Why must Step 3 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 3 happen before Step 8?\n2. Why must Step 3 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 8 because the mussels need to be divided into portions and placed on the foil before wrapping them tightly for cooking.\"\n}", "relevant_step_nos": [3, 8]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 15, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 6 because the mussels need to be divided into portions and placed on the foil before the curry mixture is added.\" \n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 16, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 because the curry mixture needs to be prepared before it can be dotted on the mussels.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 18, "title": "curried bbq mussels", "question_idx": 17, "steps": ["Preheat barbecue for medium-high heat and lightly oil the cooking grate.", "Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.", "Arrange four large sheets of foil on a flat surface.", "Divide the mussels into four even portions and place one portion on each piece of foil.", "Dot the mussels with the curry mixture.", "Sprinkle the red pepper and parsley over the top of each portion.", "Top each with lime slices.", "Wrap foil tightly around the portions.", "Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.", "Discard any mussels which do not open.", "Transfer the mussels to small bowls and garnish each with a lime wedge to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 11?", "why_question": "Why must Step 3 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: curried bbq mussels\n\nProcedure:\n1. Preheat barbecue for medium-high heat and lightly oil the cooking grate.\n2. Whisk together the butter, garlic, curry powder, cumin and salt in a small bowl.\n3. Arrange four large sheets of foil on a flat surface.\n4. Divide the mussels into four even portions and place one portion on each piece of foil.\n5. Dot the mussels with the curry mixture.\n6. Sprinkle the red pepper and parsley over the top of each portion.\n7. Top each with lime slices.\n8. Wrap foil tightly around the portions.\n9. Cook the packets on the preheated barbecue until the mussels have opened, 5 to 10 minutes.\n10. Discard any mussels which do not open.\n11. Transfer the mussels to small bowls and garnish each with a lime wedge to serve.\n\n\n1. Must Step 3 happen before Step 11?\n2. Why must Step 3 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 is necessary to properly prepare the mussels for cooking in the foil packets before they are transferred to small bowls for serving in Step 11.\"\n}", "relevant_step_nos": [3, 11]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 0, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the tart needs to be well set before serving topped with whipped cream.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 1, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the sliced strawberries need to be well coated before pouring into the base.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 2, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 6?", "why_question": "Why must Step 4 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 4 happen before Step 6?\n2. Why must Step 4 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 6 because the strawberries need to be well coated before chilling until well set.\"\n}", "relevant_step_nos": [4, 6]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 3, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because the mixture needs to be boiled before pouring it into the biscuit base.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 4, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 3 because boiling the mixture is necessary before removing it from the heat.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 5, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because boiling the mixture for one minute helps to thicken it before pouring it into the base and chilling it.\" \n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 6, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 3 because the ingredients need to be mixed and boiled before being removed from heat.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 7, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 2 because the ingredients need to be mixed together before boiling.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 8, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 6 to allow the strawberry mixture to set properly before serving.\"\n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 19, "title": "easy strawberry tart", "question_idx": 9, "steps": ["Mix together the sugar, water, jam and cornflour in a medium large saucepan.", "Boil for one minute.", "Remove from heat.", "Stir in the sliced strawberries, and toss until well coated.", "Pour into the digestive biscuit base.", "Chill until well set.", "Serve topped with whipped cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy strawberry tart\n\nProcedure:\n1. Mix together the sugar, water, jam and cornflour in a medium large saucepan.\n2. Boil for one minute.\n3. Remove from heat.\n4. Stir in the sliced strawberries, and toss until well coated.\n5. Pour into the digestive biscuit base.\n6. Chill until well set.\n7. Serve topped with whipped cream.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 because the mixture needs to be removed from heat before adding it to the biscuit base.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 20, "title": "easy-coconut-rice", "question_idx": 0, "steps": ["Combine rice with the coconut milk, water, salt, sugar and cinnamon stick in a large saucepan with a tight-fitting lid.", "Bring to the boil; reduce the heat to a simmer and cover.", "Cook until the liquid is absorbed and the rice is tender, about 25 minutes.", "Remove cinnamon stick to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: easy-coconut-rice\n\nProcedure:\n1. Combine rice with the coconut milk, water, salt, sugar and cinnamon stick in a large saucepan with a tight-fitting lid.\n2. Bring to the boil; reduce the heat to a simmer and cover.\n3. Cook until the liquid is absorbed and the rice is tender, about 25 minutes.\n4. Remove cinnamon stick to serve.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 because the cinnamon stick needs to be removed before serving the coconut rice.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 0, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 can happen before or after Step 6, as brushing with olive oil and sprinkling almonds are independent of each other.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 1, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 8 because brushing with olive oil is necessary before grilling the spinach rolls to achieve the desired texture and flavor.\"\n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 2, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 9?", "why_question": "Why must Step 6 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 6 happen before Step 9?\n2. Why must Step 6 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 9 because the spinach rolls need to cool for 10 minutes after grilling.\"\n}", "relevant_step_nos": [6, 9]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 3, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 7 can happen before Step 5 as the almonds can be sprinkled on top or rolled in after the cheese mix is wrapped in the spinach leaf.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 4, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 9?", "why_question": "Why must Step 5 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 5 happen before Step 9?\n2. Why must Step 5 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 9 because the spinach rolls need to be grilled before they can cool for 10 minutes.\"\n}", "relevant_step_nos": [5, 9]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 5, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 3 because mixing the ingredients together is necessary before stirring until creamy.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 6, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because the cheese mix needs to be prepared before it can be wrapped in the spinach leaves and brushed with olive oil.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 7, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 7 because the cheese mix needs to be creamy before sprinkling almonds on top or rolling them in the almonds.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 8, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because the spinach rolls need to be blanched before grilling and then allowed to cool before serving.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 9, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because blanching the spinach leaves makes them pliable and easier to wrap around the cheese mix.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 21, "title": "feta-and-sundried-tomato-spinach-rolls", "question_idx": 10, "steps": ["Preheat grill.", "Mix together cream cheese, feta, sundried tomatoes, basil and garlic.", "Stir until creamy, with no lumps.", "Blanch the spinach leaves.", "Wrap a dollop of cheese mix in each spinach leaf.", "Brush with olive oil.", "Sprinkle almonds on the top or alternately roll them in the almonds", "Grill spinach rolls until slightly crispy on top.", "Let cool for 10 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: feta-and-sundried-tomato-spinach-rolls\n\nProcedure:\n1. Preheat grill.\n2. Mix together cream cheese, feta, sundried tomatoes, basil and garlic.\n3. Stir until creamy, with no lumps.\n4. Blanch the spinach leaves.\n5. Wrap a dollop of cheese mix in each spinach leaf.\n6. Brush with olive oil.\n7. Sprinkle almonds on the top or alternately roll them in the almonds\n8. Grill spinach rolls until slightly crispy on top.\n9. Let cool for 10 minutes.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 blanching the spinach leaves must happen before Step 8 grilling the spinach rolls to ensure that the spinach is cooked and tender before grilling.\"\n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 22, "title": "fiery-bean-and-avocado-dip", "question_idx": 0, "steps": ["In a large bowl, mix blackeyed beans, tomatoes, red onion, Scotch bonnet chillies, Italian dressing and cayenne pepper.", "Stir in the avocados just before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fiery-bean-and-avocado-dip\n\nProcedure:\n1. In a large bowl, mix blackeyed beans, tomatoes, red onion, Scotch bonnet chillies, Italian dressing and cayenne pepper.\n2. Stir in the avocados just before serving.\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The avocados should be stirred in just before serving to prevent them from browning.\" \n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 23, "title": "frugal-beef-stew", "question_idx": 0, "steps": ["Add the onion and beef to a frying pan with a little oil (if you feel it requires it, depends on how good your pan is) and cook until the onions are soft.", "Tip out the onion/beef into a casserole dish and add the rest of the ingredients to the frying pan with half the stock, cooking on a high heat for about 10 minutes.", "Pour the contents of the frying pan into the casserole dish along with the thyme and rest of the stock (and a little bit of pepper if you like) and put on the lid.", "Cook in the oven on a low heat (about 140 C) for around 3-4 hours, giving it a stir every hour."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: frugal-beef-stew\n\nProcedure:\n1. Add the onion and beef to a frying pan with a little oil (if you feel it requires it, depends on how good your pan is) and cook until the onions are soft.\n2. Tip out the onion/beef into a casserole dish and add the rest of the ingredients to the frying pan with half the stock, cooking on a high heat for about 10 minutes.\n3. Pour the contents of the frying pan into the casserole dish along with the thyme and rest of the stock (and a little bit of pepper if you like) and put on the lid.\n4. Cook in the oven on a low heat (about 140 C) for around 3-4 hours, giving it a stir every hour.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because the ingredients need to be combined and the casserole dish needs to be covered before cooking in the oven.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 0, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 17?", "why_question": "Why must Step 5 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 5 happen before Step 17?\n2. Why must Step 5 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 17 because the tomatoes need to be processed and added to the saucepan before the meatballs are added to the tomato sauce.\"\n}", "relevant_step_nos": [5, 17]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 1, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 18?", "why_question": "Why must Step 5 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 5 happen before Step 18?\n2. Why must Step 5 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The tomatoes need to be processed and added to the saucepan before the meatballs are added to the tomato sauce.\"\n}", "relevant_step_nos": [5, 18]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 2, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The tomatoes need to be processed before being added to the saucepan.\"\n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 3, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The tomatoes need to be processed before being brought to a boil.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 4, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 15 happen before Step 20?", "why_question": "Why must Step 15 happen before Step 20?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 15 happen before Step 20?\n2. Why must Step 15 happen before Step 20?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the pasta.\"\n}", "relevant_step_nos": [15, 20]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 5, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 15 happen before Step 21?", "why_question": "Why must Step 15 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 15 happen before Step 21?\n2. Why must Step 15 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the sauce and served with the pasta.\"\n}", "relevant_step_nos": [15, 21]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 6, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 20?", "why_question": "Why must Step 11 happen before Step 20?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 11 happen before Step 20?\n2. Why must Step 11 happen before Step 20?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the pasta.\"\n}", "relevant_step_nos": [11, 20]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 7, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 21?", "why_question": "Why must Step 11 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 11 happen before Step 21?\n2. Why must Step 11 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 11 does not need to happen before Step 21 because the pasta can be cooked separately while the meatballs are being prepared and cooked.\"\n}", "relevant_step_nos": [11, 21]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 8, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 18?", "why_question": "Why must Step 11 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 11 happen before Step 18?\n2. Why must Step 11 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be shaped before they can be cooked in the frying pan and added to the tomato sauce.\"\n}", "relevant_step_nos": [11, 18]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 9, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 16 happen before Step 17?", "why_question": "Why must Step 16 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 16 happen before Step 17?\n2. Why must Step 16 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be browned before being added to the tomato sauce to ensure they are fully cooked and have a nice texture.\"\n}", "relevant_step_nos": [16, 17]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 10, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 16 happen before Step 18?", "why_question": "Why must Step 16 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 16 happen before Step 18?\n2. Why must Step 16 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be browned before being added to the tomato sauce to ensure they are fully cooked and have a nice texture.\"\n}", "relevant_step_nos": [16, 18]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 11, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 21?", "why_question": "Why must Step 3 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 3 happen before Step 21?\n2. Why must Step 3 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 3 does not need to happen before Step 21 because they are independent of each other and can happen in any order.\"\n}", "relevant_step_nos": [3, 21]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 12, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 15?", "why_question": "Why must Step 3 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 3 happen before Step 15?\n2. Why must Step 3 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 transfers some of the onion mixture to the bowl to be used in making the meatballs, which must happen before cooking the meatballs in Step 15.\"\n}", "relevant_step_nos": [3, 15]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 13, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 20?", "why_question": "Why must Step 3 happen before Step 20?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 3 happen before Step 20?\n2. Why must Step 3 happen before Step 20?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 20 because the meatballs need to be prepared and cooked before being added to the pasta and sauce.\"\n}", "relevant_step_nos": [3, 20]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 14, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 18?", "why_question": "Why must Step 3 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 3 happen before Step 18?\n2. Why must Step 3 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be shaped before they can be cooked in the sauce in Step 18.\"\n}", "relevant_step_nos": [3, 18]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 15, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 13?", "why_question": "Why must Step 3 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 3 happen before Step 13?\n2. Why must Step 3 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 transfers some of the onion mixture to the bowl to be used in making the meatballs in Step 12.\"\n}", "relevant_step_nos": [3, 13]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 16, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 14 happen before Step 16?", "why_question": "Why must Step 14 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 14 happen before Step 16?\n2. Why must Step 14 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen before Step 16 because the meatballs need to be cooked before being drained on kitchen paper.\"\n}", "relevant_step_nos": [14, 16]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 17, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 14 happen before Step 21?", "why_question": "Why must Step 14 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 14 happen before Step 21?\n2. Why must Step 14 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 14 does not need to happen before Step 21 because the pasta can be cooked and tossed with the sauce while the meatballs are still cooking in the sauce.\"\n}", "relevant_step_nos": [14, 21]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 18, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 14 happen before Step 17?", "why_question": "Why must Step 14 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 14 happen before Step 17?\n2. Why must Step 14 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be browned before being added to the tomato sauce to ensure they are fully cooked and have a nice texture.\"\n}", "relevant_step_nos": [14, 17]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 19, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 21?", "why_question": "Why must Step 13 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 13 happen before Step 21?\n2. Why must Step 13 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the pasta and sauce.\"\n}", "relevant_step_nos": [13, 21]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 20, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 15?", "why_question": "Why must Step 13 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 13 happen before Step 15?\n2. Why must Step 13 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be shaped before they can be cooked.\"\n}", "relevant_step_nos": [13, 15]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 21, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 17?", "why_question": "Why must Step 13 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 13 happen before Step 17?\n2. Why must Step 13 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be shaped before they can be cooked and added to the sauce.\"\n}", "relevant_step_nos": [13, 17]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 22, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 18?", "why_question": "Why must Step 13 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 13 happen before Step 18?\n2. Why must Step 13 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the tomato sauce in Step 18.\"\n}", "relevant_step_nos": [13, 18]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 23, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 10?", "why_question": "Why must Step 8 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 8 happen before Step 10?\n2. Why must Step 8 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The basil and oregano need to be added halfway through the simmering time, which is before the meatballs are shaped in Step 13.\"\n}", "relevant_step_nos": [8, 10]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 24, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 13?", "why_question": "Why must Step 4 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 4 happen before Step 13?\n2. Why must Step 4 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 transfers some of the onion mixture to a bowl, which is needed for Step 13 to mix with other ingredients for the meatballs.\"\n}", "relevant_step_nos": [4, 13]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 25, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 16?", "why_question": "Why must Step 4 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 4 happen before Step 16?\n2. Why must Step 4 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 is necessary to transfer the onion mixture to the large bowl before cooking the meatballs.\"\n}", "relevant_step_nos": [4, 16]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 26, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 20?", "why_question": "Why must Step 6 happen before Step 20?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 6 happen before Step 20?\n2. Why must Step 6 happen before Step 20?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 20 because the pasta needs to be cooked and tossed with the sauce before adding the meatballs and remaining sauce over the top.\"\n}", "relevant_step_nos": [6, 20]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 27, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 9?", "why_question": "Why must Step 6 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 6 happen before Step 9?\n2. Why must Step 6 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 9 because the tomatoes need to be processed and added to the saucepan before covering and simmering for 30 minutes.\"\n}", "relevant_step_nos": [6, 9]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 28, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 17?", "why_question": "Why must Step 6 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 6 happen before Step 17?\n2. Why must Step 6 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The tomatoes need to be processed and added to the saucepan before the meatballs are added to the tomato sauce.\"\n}", "relevant_step_nos": [6, 17]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 29, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 20?", "why_question": "Why must Step 7 happen before Step 20?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 7 happen before Step 20?\n2. Why must Step 7 happen before Step 20?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The tomato sauce needs to be ready before the pasta is cooked and tossed in the sauce.\"\n}", "relevant_step_nos": [7, 20]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 30, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 17?", "why_question": "Why must Step 7 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 7 happen before Step 17?\n2. Why must Step 7 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The tomato sauce needs to be brought to a boil before adding the meatballs.\"\n}", "relevant_step_nos": [7, 17]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 31, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 20?", "why_question": "Why must Step 12 happen before Step 20?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 12 happen before Step 20?\n2. Why must Step 12 happen before Step 20?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be cooked before being added to the pasta.\"\n}", "relevant_step_nos": [12, 20]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 32, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 18?", "why_question": "Why must Step 12 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 12 happen before Step 18?\n2. Why must Step 12 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The meatballs need to be shaped and cooked before being added to the tomato sauce in Step 18.\"\n}", "relevant_step_nos": [12, 18]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 33, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 21?", "why_question": "Why must Step 2 happen before Step 21?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 2 happen before Step 21?\n2. Why must Step 2 happen before Step 21?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 21 because the sauce needs to be prepared before serving the meatballs and pasta.\"\n}", "relevant_step_nos": [2, 21]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 34, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 12?", "why_question": "Why must Step 2 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 2 happen before Step 12?\n2. Why must Step 2 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The onion and garlic need to be saut\u00e9ed before being added to the meatball mixture.\"\n}", "relevant_step_nos": [2, 12]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 35, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 because the onion and garlic need to be saut\u00e9ed before transferring a portion to the large bowl in Step 3.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 36, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 13?", "why_question": "Why must Step 1 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 1 happen before Step 13?\n2. Why must Step 1 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to cook the meatballs in the frying pan.\"\n}", "relevant_step_nos": [1, 13]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 37, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 12?", "why_question": "Why must Step 1 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 1 happen before Step 12?\n2. Why must Step 1 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to cook the onion and garlic before adding the other ingredients for the meatballs.\"\n}", "relevant_step_nos": [1, 12]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 38, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to create the onion mixture for the meatballs.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 39, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 4?", "why_question": "Why must Step 1 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 1 happen before Step 4?\n2. Why must Step 1 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to saut\u00e9 the onion and garlic before transferring some of the mixture to a bowl in Step 3.\"\n}", "relevant_step_nos": [1, 4]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 40, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to prepare the saucepan for saut\u00e9ing the onion and garlic in Step 2.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 24, "title": "fusilli-and-meatballs", "question_idx": 41, "steps": ["Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.", "Saut\u00e9 the onion and garlic for about 5 minutes or until soft.", "Transfer 2 tablespoons of the onion mixture to a large bowl.", "Set aside.", "Whiz the tomatoes, with their juice, in a food processor until fairly smooth.", "Add to the saucepan.", "Bring to the boil over medium-high heat.", "Reduce the heat to medium-low.", "Cover and simmer, stirring often, for 30 minutes.", "Add the basil and oregano halfway through this time.", "Process the bread until crumbs form.", "Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.", "Shape into 20 meatballs about 2.5cm (1in) diameter.", "Coat a frying pan with cooking spray or olive oil and set over medium-high heat.", "Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.", "Drain on kitchen paper.", "Add the meatballs to the tomato sauce.", "Cover and cook, stirring occasionally, for 20 minutes.", "Meanwhile, cook the pasta according to the instructions on the packet.", "Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.", "Spoon the meatballs and remaining sauce over the top, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 18?", "why_question": "Why must Step 10 happen before Step 18?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: fusilli-and-meatballs\n\nProcedure:\n1. Coat a large saucepan with cooking spray or a teaspoon of olive oil and set over a medium-high heat.\n2. Saut\u00e9 the onion and garlic for about 5 minutes or until soft.\n3. Transfer 2 tablespoons of the onion mixture to a large bowl.\n4. Set aside.\n5. Whiz the tomatoes, with their juice, in a food processor until fairly smooth.\n6. Add to the saucepan.\n7. Bring to the boil over medium-high heat.\n8. Reduce the heat to medium-low.\n9. Cover and simmer, stirring often, for 30 minutes.\n10. Add the basil and oregano halfway through this time.\n11. Process the bread until crumbs form.\n12. Add the crumbs, beef, egg, milk and pepper to the onion mixture in the bowl and mix just until blended.\n13. Shape into 20 meatballs about 2.5cm (1in) diameter.\n14. Coat a frying pan with cooking spray or olive oil and set over medium-high heat.\n15. Cook the meatballs, in batches, for about 8 minutes or until browned on all sides.\n16. Drain on kitchen paper.\n17. Add the meatballs to the tomato sauce.\n18. Cover and cook, stirring occasionally, for 20 minutes.\n19. Meanwhile, cook the pasta according to the instructions on the packet.\n20. Drain and toss with 3 ladlefuls of the sauce in a large heated serving bowl.\n21. Spoon the meatballs and remaining sauce over the top, then serve.\n\n\n1. Must Step 10 happen before Step 18?\n2. Why must Step 10 happen before Step 18?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 does not need to happen before Step 18 because the meatballs are added to the tomato sauce in Step 17, not Step 18.\"\n}", "relevant_step_nos": [10, 18]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 0, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to prepare the rice before bringing it to a boil in Step 2.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 1, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 needs to happen before Step 8 because the sesame oil needs to be heated before the tofu is fried and seasoned with salt.\"\n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 2, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 12?", "why_question": "Why must Step 5 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 5 happen before Step 12?\n2. Why must Step 5 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 12 because the green curry sauce needs to be drizzled over the tofu and rice, and the sauce is prepared in Step 9 and 10, which requires the use of a separate medium saucepan.\" \n}", "relevant_step_nos": [5, 12]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 3, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 12?", "why_question": "Why must Step 11 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 11 happen before Step 12?\n2. Why must Step 11 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 12 because the green curry paste needs to be mixed into the coconut milk and simmered before being drizzled over the tofu and rice to serve.\"\n}", "relevant_step_nos": [11, 12]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 4, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 12?", "why_question": "Why must Step 8 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 8 happen before Step 12?\n2. Why must Step 8 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 12 because the tofu needs to be seasoned with salt before being served with the green curry.\" \n}", "relevant_step_nos": [8, 12]}
{"plan_idx": 25, "title": "green-tofu-curry", "question_idx": 5, "steps": ["Place water in a medium saucepan, and stir in the rice.", "Bring to the boil.", "Cover, reduce heat, and simmer 20 minutes.", "Remove from heat, cool slightly, and fluff with a fork.", "Heat the sesame oil in a separate medium saucepan over medium heat.", "Stir in tofu.", "Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.", "Season with salt.", "In a small saucepan, bring the coconut milk to the boil.", "Mix in green curry paste.", "Reduce heat, and simmer 5 minutes.", "Drizzle generously over the tofu and rice to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 10?", "why_question": "Why must Step 9 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: green-tofu-curry\n\nProcedure:\n1. Place water in a medium saucepan, and stir in the rice.\n2. Bring to the boil.\n3. Cover, reduce heat, and simmer 20 minutes.\n4. Remove from heat, cool slightly, and fluff with a fork.\n5. Heat the sesame oil in a separate medium saucepan over medium heat.\n6. Stir in tofu.\n7. Stirring occasionally, fry about 20 minutes, until evenly crisp and lightly browned.\n8. Season with salt.\n9. In a small saucepan, bring the coconut milk to the boil.\n10. Mix in green curry paste.\n11. Reduce heat, and simmer 5 minutes.\n12. Drizzle generously over the tofu and rice to serve.\n\n\n1. Must Step 9 happen before Step 10?\n2. Why must Step 9 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 10 because the coconut milk needs to be brought to a boil before mixing in the green curry paste.\"\n}", "relevant_step_nos": [9, 10]}
{"plan_idx": 26, "title": "halloumi-and-figs", "question_idx": 0, "steps": ["Preheat a grill for high heat.", "Break up the halloumi cheese, and toss with the fresh figs and walnuts in a small dish.", "Grill under the preheated grill until the halloumi is browned and the figs begin to caramelise, about 3 minutes.", "Drizzle with the honey."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: halloumi-and-figs\n\nProcedure:\n1. Preheat a grill for high heat.\n2. Break up the halloumi cheese, and toss with the fresh figs and walnuts in a small dish.\n3. Grill under the preheated grill until the halloumi is browned and the figs begin to caramelise, about 3 minutes.\n4. Drizzle with the honey.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 because the halloumi cheese, figs, and walnuts need to be grilled before drizzling with honey.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 0, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 because the flour needs to be folded into the dough before forming it into small balls.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 1, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because the dough needs to be chilled and formed into balls before making the hollows and baking.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 2, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because the dough needs to be chilled and formed into balls before making a hollow and filling with jam.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 3, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 5?", "why_question": "Why must Step 1 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 1 happen before Step 5?\n2. Why must Step 1 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 5 because the biscuit dough needs to be prepared and chilled before forming into balls and making the hollows for the jam.\" \n}", "relevant_step_nos": [1, 5]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 4, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 3 because beating the butter and adding the sugar, vanilla sugar, and eggs is the first step in preparing the dough for the biscuits.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 27, "title": "jam-drop-biscuits", "question_idx": 5, "steps": ["Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.", "Sieve in flour and fold in.", "Chill for at least 30 minutes.", "Form small balls from the biscuit dough and roll in ground hazelnuts.", "Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.", "Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: jam-drop-biscuits\n\nProcedure:\n1. Beat butter until fluffy and gradually add the sugar, vanilla sugar and eggs.\n2. Sieve in flour and fold in.\n3. Chill for at least 30 minutes.\n4. Form small balls from the biscuit dough and roll in ground hazelnuts.\n5. Make a hollow into each ball with the end of a wooden spoon and place onto a baking tray lined with baking parchment.\n6. Fill the holes with jam and bake in a preheated oven at 170 C / Gas 3 for 20 minutes.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the hollows for the jam need to be made before baking the biscuits.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 29, "title": "lamb-curry--serves-4-", "question_idx": 0, "steps": ["Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.", "Fry the lamb in a little oil, then add the remaining onion.", "Gently fry for a minute, then add the paste and cook for a couple of minutes.", "Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.", "Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lamb-curry--serves-4-\n\nProcedure:\n1. Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.\n2. Fry the lamb in a little oil, then add the remaining onion.\n3. Gently fry for a minute, then add the paste and cook for a couple of minutes.\n4. Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.\n5. Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the ingredients added in Step 4 need time to simmer and become tender before adding the fresh tomatoes and spinach in Step 5.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 29, "title": "lamb-curry--serves-4-", "question_idx": 1, "steps": ["Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.", "Fry the lamb in a little oil, then add the remaining onion.", "Gently fry for a minute, then add the paste and cook for a couple of minutes.", "Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.", "Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lamb-curry--serves-4-\n\nProcedure:\n1. Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.\n2. Fry the lamb in a little oil, then add the remaining onion.\n3. Gently fry for a minute, then add the paste and cook for a couple of minutes.\n4. Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.\n5. Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the flavors of the curry paste to infuse with the lamb and other ingredients.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 29, "title": "lamb-curry--serves-4-", "question_idx": 2, "steps": ["Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.", "Fry the lamb in a little oil, then add the remaining onion.", "Gently fry for a minute, then add the paste and cook for a couple of minutes.", "Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.", "Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lamb-curry--serves-4-\n\nProcedure:\n1. Blend the garlic, ginger and one of the onions in a food processor then mix with the curry paste.\n2. Fry the lamb in a little oil, then add the remaining onion.\n3. Gently fry for a minute, then add the paste and cook for a couple of minutes.\n4. Add the water, tin of tomatoes, sugar, mushrooms and potatoes, and gently simmer covered for a few hours, until everything is tender.\n5. Add the two fresh tomatoes and the spinach and cook for a couple of minutes, then serve.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 because the lamb needs to be fried before adding the other ingredients and simmering.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 0, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 16?", "why_question": "Why must Step 4 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 4 happen before Step 16?\n2. Why must Step 4 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 16 because the dough needs to be prepared and baked before assembling the shortcakes.\"\n}", "relevant_step_nos": [4, 16]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 1, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the dough needs to be formed before it can be kneaded.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 2, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because the dough needs to be prepared and baked before the shortcakes can be assembled.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 3, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 16?", "why_question": "Why must Step 3 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 3 happen before Step 16?\n2. Why must Step 3 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 16 because the lemon zest is incorporated into the dough in Step 4, and the lemon butter is made in Step 11, so the lemon flavor needs to be present in the shortcakes before assembling.\" \n}", "relevant_step_nos": [3, 16]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 4, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 7 because rubbing the butter into the flour is necessary to create the dough for the shortcakes before dividing and shaping it in Step 7.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 5, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because rubbing the butter into the flour is part of the process of making the dough for the shortcakes.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 6, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 10?", "why_question": "Why must Step 3 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 3 happen before Step 10?\n2. Why must Step 3 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 10 because the lemon zest is used in both steps, and it needs to be incorporated into the dough before being used in the lemon butter.\"\n}", "relevant_step_nos": [3, 10]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 7, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 16?", "why_question": "Why must Step 12 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 12 happen before Step 16?\n2. Why must Step 12 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen before Step 16 because the whipped cream needs to be prepared before assembling the shortcakes.\"\n}", "relevant_step_nos": [12, 16]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 8, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 10?", "why_question": "Why must Step 7 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 7 happen before Step 10?\n2. Why must Step 7 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 10 because the shortcakes need to be baked before they can be cooled for 15 minutes.\"\n}", "relevant_step_nos": [7, 10]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 9, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 15?", "why_question": "Why must Step 7 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 7 happen before Step 15?\n2. Why must Step 7 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The dough needs to be divided and shaped into circles before assembling the shortcakes.\"\n}", "relevant_step_nos": [7, 15]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 10, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the dough needs to be kneaded before it is divided in half.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 11, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 9?", "why_question": "Why must Step 5 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 5 happen before Step 9?\n2. Why must Step 5 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 is kneading the dough, which needs to happen before baking in Step 9.\"\n}", "relevant_step_nos": [5, 9]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 12, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 7 because the dough needs to be kneaded before it is divided and shaped into circles.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 13, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 15?", "why_question": "Why must Step 5 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 5 happen before Step 15?\n2. Why must Step 5 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 15 because the dough needs to be kneaded and prepared before assembling the shortcakes.\"\n}", "relevant_step_nos": [5, 15]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 14, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 14?", "why_question": "Why must Step 2 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 2 happen before Step 14?\n2. Why must Step 2 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 14 because the shortcakes need to be prepared before assembling them with the strawberries and whipped cream.\"\n}", "relevant_step_nos": [2, 14]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 15, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 8?", "why_question": "Why must Step 2 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 2 happen before Step 8?\n2. Why must Step 2 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 8 because the dough needs to be prepared before it can be baked.\"\n}", "relevant_step_nos": [2, 8]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 16, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 3 because the dry ingredients need to be combined before rubbing the butter into the mixture.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 17, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 10?", "why_question": "Why must Step 2 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 2 happen before Step 10?\n2. Why must Step 2 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 10 because the shortcakes need to be baked before the lemon butter can be prepared.\"\n}", "relevant_step_nos": [2, 10]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 18, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 9?", "why_question": "Why must Step 2 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 2 happen before Step 9?\n2. Why must Step 2 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 9 because the dough needs to be prepared and baked before the strawberries are added to the shortcakes.\"\n}", "relevant_step_nos": [2, 9]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 19, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 16?", "why_question": "Why must Step 2 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 2 happen before Step 16?\n2. Why must Step 2 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 provides the instructions for making the shortcake, which is necessary before assembling the lemon-zested-strawberry-shortcakes in Step 16.\"\n}", "relevant_step_nos": [2, 16]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 20, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because the flour mixture needs to be combined and prepared before it can be divided and shaped in Step 6.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 21, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 16?", "why_question": "Why must Step 6 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 6 happen before Step 16?\n2. Why must Step 6 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 16 because the dough needs to be divided and shaped into circles before baking and assembling the shortcakes.\"\n}", "relevant_step_nos": [6, 16]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 22, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 15?", "why_question": "Why must Step 6 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 6 happen before Step 15?\n2. Why must Step 6 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 divides the dough into two portions which are then used to make the shortcakes, so it must happen before assembling the shortcakes in Step 15.\"\n}", "relevant_step_nos": [6, 15]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 23, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 divides the dough in half, which is necessary before patting or rolling each half into a circle in Step 7.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 24, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 16?", "why_question": "Why must Step 13 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 13 happen before Step 16?\n2. Why must Step 13 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 16 because the whipped cream needs to be prepared before assembling the shortcakes.\"\n}", "relevant_step_nos": [13, 16]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 25, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 15?", "why_question": "Why must Step 13 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 13 happen before Step 15?\n2. Why must Step 13 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 15 because the whipped cream needs to be prepared before assembling the shortcakes.\"\n}", "relevant_step_nos": [13, 15]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 26, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 15?", "why_question": "Why must Step 9 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 9 happen before Step 15?\n2. Why must Step 9 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 15 because the shortcakes need to be baked and cooled before assembling the dessert.\"\n}", "relevant_step_nos": [9, 15]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 27, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 10?", "why_question": "Why must Step 9 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 9 happen before Step 10?\n2. Why must Step 9 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The shortcakes need to cool for 15 minutes before adding the lemon butter in order to prevent the butter from melting.\" \n}", "relevant_step_nos": [9, 10]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 28, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 16?", "why_question": "Why must Step 9 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 9 happen before Step 16?\n2. Why must Step 9 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The shortcakes need to be baked and cooled before assembling the dessert.\"\n}", "relevant_step_nos": [9, 16]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 29, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 14 happen before Step 15?", "why_question": "Why must Step 14 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 14 happen before Step 15?\n2. Why must Step 14 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen before Step 15 because the shortcakes need to be split in half before assembling them with lemon butter, strawberries, and whipped cream.\"\n}", "relevant_step_nos": [14, 15]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 30, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 14 happen before Step 16?", "why_question": "Why must Step 14 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 14 happen before Step 16?\n2. Why must Step 14 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen before Step 16 because the shortcakes need to be split in half before assembling them with the strawberries and whipped cream.\"\n}", "relevant_step_nos": [14, 16]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 31, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 16?", "why_question": "Why must Step 1 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 1 happen before Step 16?\n2. Why must Step 1 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The strawberries need to be refrigerated until serving, so they must be prepared before assembling the shortcakes in Step 16.\"\n}", "relevant_step_nos": [1, 16]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 32, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 14?", "why_question": "Why must Step 8 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 8 happen before Step 14?\n2. Why must Step 8 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The shortcakes need to be baked before they can be split in half for assembly.\"\n}", "relevant_step_nos": [8, 14]}
{"plan_idx": 30, "title": "lemon-zested-strawberry-shortcakes", "question_idx": 33, "steps": ["In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.", "In a bowl, combine the flour, sugar, baking powder and salt.", "Rub the butter into the flour until the mixture resembles coarse crumbs.", "Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).", "Turn onto a lightly floured surface; knead 10 times.", "Divide dough in half.", "Gently pat or roll each half into a 1 .75cm thick circle.", "Place 5cm apart on an ungreased baking tray.", "Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.", "Remove to a wire rack; cool for 15 minutes.", "Meanwhile, in a small bowl, combine butter and lemon zest; set aside.", "In a small mixing bowl, beat cream until it begins to thicken.", "Add sugar; beat until stiff peaks form.", "To assemble, split shortcakes in half.", "Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.", "Cover with shortcake top and remaining berries and cream."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 16?", "why_question": "Why must Step 8 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lemon-zested-strawberry-shortcakes\n\nProcedure:\n1. In a small bowl, combine strawberries and sugar; cover and refrigerate until serving.\n2. In a bowl, combine the flour, sugar, baking powder and salt.\n3. Rub the butter into the flour until the mixture resembles coarse crumbs.\n4. Combine the milk, egg yolk and lemon zest; stir into crumb mixture until a soft dough forms (dough will be sticky).\n5. Turn onto a lightly floured surface; knead 10 times.\n6. Divide dough in half.\n7. Gently pat or roll each half into a 1 .75cm thick circle.\n8. Place 5cm apart on an ungreased baking tray.\n9. Bake at 200 C / Gas 6 for 8-10 minutes or until golden brown.\n10. Remove to a wire rack; cool for 15 minutes.\n11. Meanwhile, in a small bowl, combine butter and lemon zest; set aside.\n12. In a small mixing bowl, beat cream until it begins to thicken.\n13. Add sugar; beat until stiff peaks form.\n14. To assemble, split shortcakes in half.\n15. Place cake bottoms on dessert plates; spread with lemon butter, then top each with a 1/4 of the strawberries and whipped cream.\n16. Cover with shortcake top and remaining berries and cream.\n\n\n1. Must Step 8 happen before Step 16?\n2. Why must Step 8 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The shortcakes need to be baked before they can be assembled with the strawberries and whipped cream.\"\n}", "relevant_step_nos": [8, 16]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 0, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 11?", "why_question": "Why must Step 1 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 1 happen before Step 11?\n2. Why must Step 1 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 11 because the oil needs to be heated before the soup can be reheated until just bubbling.\"\n}", "relevant_step_nos": [1, 11]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 1, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 9?", "why_question": "Why must Step 1 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 1 happen before Step 9?\n2. Why must Step 1 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 9 because the soup needs to be heated and cooked before blending.\"\n}", "relevant_step_nos": [1, 9]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 2, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 8 because the bay leaf needs to be removed from the soup before blending it.\"\n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 3, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 9?", "why_question": "Why must Step 5 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 5 happen before Step 9?\n2. Why must Step 5 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 9 because the lentils, stock, tomatoes, and other ingredients need to be blended together before checking the consistency and blending the soup.\" \n}", "relevant_step_nos": [5, 9]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 4, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 9?", "why_question": "Why must Step 6 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 6 happen before Step 9?\n2. Why must Step 6 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 9 because the soup needs to be cooked and softened before blending.\"\n}", "relevant_step_nos": [6, 9]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 5, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 11?", "why_question": "Why must Step 6 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 6 happen before Step 11?\n2. Why must Step 6 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 11 because the soup needs to simmer and cook for 25-30 minutes before being seasoned and reheated.\"\n}", "relevant_step_nos": [6, 11]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 6, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 12?", "why_question": "Why must Step 6 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 6 happen before Step 12?\n2. Why must Step 6 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 12 because the soup needs to be simmered until the lentils and vegetables are very soft before serving.\"\n}", "relevant_step_nos": [6, 12]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 7, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because the onion needs to soften before adding the other ingredients.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 8, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 11?", "why_question": "Why must Step 2 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 2 happen before Step 11?\n2. Why must Step 2 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 11 because the onions need to soften before the soup is seasoned and reheated.\"\n}", "relevant_step_nos": [2, 11]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 9, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 12?", "why_question": "Why must Step 2 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 2 happen before Step 12?\n2. Why must Step 2 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 12 because the onions need to be softened before serving the soup.\"\n}", "relevant_step_nos": [2, 12]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 10, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 12?", "why_question": "Why must Step 11 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 11 happen before Step 12?\n2. Why must Step 11 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 12 to ensure that the soup is seasoned to taste before serving.\"\n}", "relevant_step_nos": [11, 12]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 11, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 12?", "why_question": "Why must Step 4 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 4 happen before Step 12?\n2. Why must Step 4 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 12 because the spices need to be cooked and incorporated into the soup before serving.\"\n}", "relevant_step_nos": [4, 12]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 12, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because the spices and garlic need to be cooked with the vegetables before blending the soup.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 13, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 11?", "why_question": "Why must Step 4 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 4 happen before Step 11?\n2. Why must Step 4 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 11 because the spices need to be cooked to release their flavors before the soup is seasoned to taste.\"\n}", "relevant_step_nos": [4, 11]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 14, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 11?", "why_question": "Why must Step 9 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 9 happen before Step 11?\n2. Why must Step 9 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 11 because blending the soup in Step 9 affects its consistency, which needs to be checked and adjusted in Step 10 before reheating in Step 11.\"\n}", "relevant_step_nos": [9, 11]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 15, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 12?", "why_question": "Why must Step 9 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 9 happen before Step 12?\n2. Why must Step 9 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 is necessary to blend the soup before serving it in Step 12.\"\n}", "relevant_step_nos": [9, 12]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 16, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 10?", "why_question": "Why must Step 9 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 9 happen before Step 10?\n2. Why must Step 9 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 10 because the consistency of the soup needs to be checked and adjusted before seasoning and reheating.\"\n}", "relevant_step_nos": [9, 10]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 17, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 6 because the vegetables need to be cooked before simmering the soup for 25-30 minutes.\"\n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 18, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 12?", "why_question": "Why must Step 3 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 3 happen before Step 12?\n2. Why must Step 3 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 12 because the vegetables need to be cooked before serving the soup.\"\n}", "relevant_step_nos": [3, 12]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 19, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the bay leaf is removed before blending the soup.\"\n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 31, "title": "lightly spiced tomato and red lentil soup", "question_idx": 20, "steps": ["Heat the oil in a saucepan.", "Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.", "Stir in the celery and carrot and cook for 3 minutes, stirring frequently.", "Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.", "Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.", "Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.", "Meanwhile, stir the coriander into the yoghurt for the garnish.", "Remove the bay leaf from the soup.", "Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.", "Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.", "Season to taste, then reheat until just bubbling.", "Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 10?", "why_question": "Why must Step 8 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: lightly spiced tomato and red lentil soup\n\nProcedure:\n1. Heat the oil in a saucepan.\n2. Add the onion and cook over a low heat for 7-8 minutes, stirring occasionally, until beginning to soften.\n3. Stir in the celery and carrot and cook for 3 minutes, stirring frequently.\n4. Add the garlic, cumin and coriander and cook for a further minute, stirring constantly.\n5. Add the lentils, stock, tomatoes with their juice, tomato pur\u00c3\u00a9e and bay leaf.\n6. Bring to the boil, reduce the heat, then half-cover the pan with a lid and simmer for 25-30 minutes until the lentils and vegetables are very soft.\n7. Meanwhile, stir the coriander into the yoghurt for the garnish.\n8. Remove the bay leaf from the soup.\n9. Blend the soup in the pan using a hand-held blender, or tip into a blender or food processor, process until smooth, then return the soup to the pan.\n10. Check the consistency; it will be fairly thick, so if you prefer it thinner, dilute with a little more stock.\n11. Season to taste, then reheat until just bubbling.\n12. Serve drizzled with some of the coriander yoghurt and sprinkled with freshly ground black pepper.\n\n\n1. Must Step 8 happen before Step 10?\n2. Why must Step 8 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 10 to ensure that the bay leaf is removed from the soup before checking the consistency and adjusting it with more stock if needed.\"\n}", "relevant_step_nos": [8, 10]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 0, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 7?", "why_question": "Why must Step 4 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 4 happen before Step 7?\n2. Why must Step 4 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 7 because the dough needs to be formed before placing it on the baking tray.\"\n}", "relevant_step_nos": [4, 7]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 1, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 8 because the dough needs to be prepared before baking.\" \n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 2, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 9?", "why_question": "Why must Step 7 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 7 happen before Step 9?\n2. Why must Step 7 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 9 because the rock cakes need to be spaced out on the baking tray before they are baked, and then they need to be removed from the tray to cool.\"\n}", "relevant_step_nos": [7, 9]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 3, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 8?", "why_question": "Why must Step 7 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 7 happen before Step 8?\n2. Why must Step 7 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 8 because the dough needs to be spaced out on the baking tray before baking.\"\n}", "relevant_step_nos": [7, 8]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 4, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the dough needs to be placed on the baking tray before being spaced out.\" \n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 5, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 8 because the dough needs to be formed into balls and placed on the baking tray before baking.\" \n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 6, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 6 because the flour, Splenda, and butter need to be mixed together to form a crumble before adding the other ingredients and forming the dough.\"\n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 7, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 because the flour and Splenda need to be mixed with the butter before adding the egg and currants to achieve the right consistency and texture for the dough.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 8, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the dough needs to be mixed before it can be formed into balls.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 9, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 9 because the rock cakes need to be baked before they can be removed and placed on a cooling tray.\"\n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 10, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 8?", "why_question": "Why must Step 1 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 1 happen before Step 8?\n2. Why must Step 1 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 8 because the oven needs to be preheated before baking the rock cakes.\"\n}", "relevant_step_nos": [1, 8]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 11, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 7?", "why_question": "Why must Step 2 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 2 happen before Step 7?\n2. Why must Step 2 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 7 to ensure that the baking tray is prepared and ready for the dough to be placed on it.\"\n}", "relevant_step_nos": [2, 7]}
{"plan_idx": 32, "title": "no-sugar-rock-cakes", "question_idx": 12, "steps": ["Preheat oven to 200 C / Gas 6.", "Line a baking tray with baking parchment.", "Mix flour, Splenda&reg;", "and butter in a bowl until a fine crumble is achieved.", "Add the egg and currants, and mix into a dough.", "Place balls of dough (slightly bigger than a golf ball) onto the baking tray;", "remember to space them out.", "Bake for 20 minutes, or till a nice golden brown.", "Remove and place on cooling tray."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 9?", "why_question": "Why must Step 2 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: no-sugar-rock-cakes\n\nProcedure:\n1. Preheat oven to 200 C / Gas 6.\n2. Line a baking tray with baking parchment.\n3. Mix flour, Splenda&reg;\n4. and butter in a bowl until a fine crumble is achieved.\n5. Add the egg and currants, and mix into a dough.\n6. Place balls of dough (slightly bigger than a golf ball) onto the baking tray;\n7. remember to space them out.\n8. Bake for 20 minutes, or till a nice golden brown.\n9. Remove and place on cooling tray.\n\n\n1. Must Step 2 happen before Step 9?\n2. Why must Step 2 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to prepare the baking tray before removing the rock cakes from the oven.\"\n}", "relevant_step_nos": [2, 9]}
{"plan_idx": 33, "title": "old-fashioned-welsh-rarebit", "question_idx": 0, "steps": ["Grate 250g well-flavoured mature Cheddar cheese (you can also use a good Double Gloucester or Shropshire) and put into a heavy-based medium-sized pan (preferably nonstick) with 10g butter, 1 tsp mustard (English, wholegrain or Dijon), plenty of ground black pepper and 4 tbsp ale or semi-skimmed milk.", "Set over a low heat and melt gently, stirring, until smooth and thick.", "Spoon onto 4 thick slices of toast and grill until browned and bubbling.", "Serve with grilled mushrooms and a leaf salad for a light lunch or snack."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: old-fashioned-welsh-rarebit\n\nProcedure:\n1. Grate 250g well-flavoured mature Cheddar cheese (you can also use a good Double Gloucester or Shropshire) and put into a heavy-based medium-sized pan (preferably nonstick) with 10g butter, 1 tsp mustard (English, wholegrain or Dijon), plenty of ground black pepper and 4 tbsp ale or semi-skimmed milk.\n2. Set over a low heat and melt gently, stirring, until smooth and thick.\n3. Spoon onto 4 thick slices of toast and grill until browned and bubbling.\n4. Serve with grilled mushrooms and a leaf salad for a light lunch or snack.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 3 because the cheese mixture needs to be melted and thickened before spooning onto the toast and grilling.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 33, "title": "old-fashioned-welsh-rarebit", "question_idx": 1, "steps": ["Grate 250g well-flavoured mature Cheddar cheese (you can also use a good Double Gloucester or Shropshire) and put into a heavy-based medium-sized pan (preferably nonstick) with 10g butter, 1 tsp mustard (English, wholegrain or Dijon), plenty of ground black pepper and 4 tbsp ale or semi-skimmed milk.", "Set over a low heat and melt gently, stirring, until smooth and thick.", "Spoon onto 4 thick slices of toast and grill until browned and bubbling.", "Serve with grilled mushrooms and a leaf salad for a light lunch or snack."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: old-fashioned-welsh-rarebit\n\nProcedure:\n1. Grate 250g well-flavoured mature Cheddar cheese (you can also use a good Double Gloucester or Shropshire) and put into a heavy-based medium-sized pan (preferably nonstick) with 10g butter, 1 tsp mustard (English, wholegrain or Dijon), plenty of ground black pepper and 4 tbsp ale or semi-skimmed milk.\n2. Set over a low heat and melt gently, stirring, until smooth and thick.\n3. Spoon onto 4 thick slices of toast and grill until browned and bubbling.\n4. Serve with grilled mushrooms and a leaf salad for a light lunch or snack.\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 3 because the cheese mixture needs to be prepared before it can be spooned onto the toast and grilled.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 0, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 8 because the stew needs to be fully cooked before serving piping hot.\"\n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 1, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the potatoes need to be cooked before seasoning the stew.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 2, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 4?", "why_question": "Why must Step 1 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 1 happen before Step 4?\n2. Why must Step 1 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 4 because the first batch of potatoes needs to cook and break up to thicken the stew before adding the remaining potatoes and carrots.\"\n}", "relevant_step_nos": [1, 4]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 3, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to cook the lamb before skimming off any fat and simmering for 30 minutes.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 4, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 3 because it involves cooking the lamb and creating a base for the stew before adding the other ingredients.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 5, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 6?", "why_question": "Why must Step 1 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 1 happen before Step 6?\n2. Why must Step 1 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 6 because the lamb needs to be cooked and seasoned before serving the stew piping hot.\"\n}", "relevant_step_nos": [1, 6]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 6, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 7?", "why_question": "Why must Step 2 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 2 happen before Step 7?\n2. Why must Step 2 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 7 because the stew needs to simmer and cook before adding the final seasoning and serving.\"\n}", "relevant_step_nos": [2, 7]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 7, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because seasoning the stew with salt and pepper should be done after the stew has been cooked and the flavors have developed.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 8, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 6?", "why_question": "Why must Step 4 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 4 happen before Step 6?\n2. Why must Step 4 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 6 because the seasoning and addition of fresh parsley or chopped leeks should be done after all the vegetables are cooked and the stew is ready to be served.\"\n}", "relevant_step_nos": [4, 6]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 9, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because seasoning the stew with salt and pepper to taste should be done before adding fresh parsley or chopped leeks.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 10, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 7 because it is necessary to thicken the stew before adding the fresh parsley or chopped leeks.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 34, "title": "one-pot irish stew", "question_idx": 11, "steps": ["Place the lamb in a large pot, cover with water and add a pinch of salt.", "Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.", "Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.", "Add the remaining potatoes and carrots.", "Simmer for another 20-30 minutes until potatoes are cooked.", "Season well with salt and pepper to taste.", "Stir in fresh parsley or chopped leeks.", "Serve piping hot."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: one-pot irish stew\n\nProcedure:\n1. Place the lamb in a large pot, cover with water and add a pinch of salt.\n2. Bring to the boil; skim off any white fat that comes to the surface, then simmer on low heat for another 30 minutes.\n3. Add the onion and half of the potatoes and cook for 30 minutes until the potatoes are soft; break them up and stir - this will thicken the stew.\n4. Add the remaining potatoes and carrots.\n5. Simmer for another 20-30 minutes until potatoes are cooked.\n6. Season well with salt and pepper to taste.\n7. Stir in fresh parsley or chopped leeks.\n8. Serve piping hot.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 because it is necessary to cook the first batch of potatoes before adding the remaining potatoes and carrots.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 0, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 10?", "why_question": "Why must Step 1 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 1 happen before Step 10?\n2. Why must Step 1 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 needs to happen before Step 10 because the onions need to be cooked before serving the dish.\"\n}", "relevant_step_nos": [1, 10]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 1, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 6?", "why_question": "Why must Step 1 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 1 happen before Step 6?\n2. Why must Step 1 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 6 because the onions need to be softened before adding the other ingredients and creating the sauce.\"\n}", "relevant_step_nos": [1, 6]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 2, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 needs to happen before Step 5 because the mushrooms need to release their juices before adding the other ingredients for the sauce.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 3, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 8?", "why_question": "Why must Step 3 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 3 happen before Step 8?\n2. Why must Step 3 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 8 because the mushrooms need to be cooked and seasoned before adding the roasted red peppers.\"\n}", "relevant_step_nos": [3, 8]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 4, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 needs to happen before Step 6 because the mushrooms need to release their juices before adding the other ingredients.\" \n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 5, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 10?", "why_question": "Why must Step 7 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 7 happen before Step 10?\n2. Why must Step 7 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 10 because the peppers need to be mixed into the mushrooms before serving.\"\n}", "relevant_step_nos": [7, 10]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 6, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 9?", "why_question": "Why must Step 7 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 7 happen before Step 9?\n2. Why must Step 7 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 9 because the peppers need to be mixed into the mushrooms before adding the tuna and parsley.\"\n}", "relevant_step_nos": [7, 9]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 7, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because the mushrooms need to give up their juices before adding the tuna and other ingredients.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 8, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 needs to happen before Step 5 to ensure the mushrooms are cooked and have released their juices before adding the other ingredients.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 9, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 8 because the mushrooms need to give up their juices before adding the roasted red peppers.\"\n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 10, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 9 because the peppers need to be mixed into the mushrooms before adding the tuna and parsley.\"\n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 11, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 10?", "why_question": "Why must Step 8 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 8 happen before Step 10?\n2. Why must Step 8 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 10 because the peppers and tuna need to be mixed into the mushrooms before serving.\"\n}", "relevant_step_nos": [8, 10]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 12, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because the onions need to be softened before adding the other ingredients to the pan.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 13, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 softens the onions before adding the other ingredients in Step 6.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 14, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 softens the onions before adding the mushrooms to ensure even cooking.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 15, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 softens the onions before adding the mushrooms, allowing for better flavor and texture in the dish.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 16, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 10?", "why_question": "Why must Step 9 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 9 happen before Step 10?\n2. Why must Step 9 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 10 because the tuna and parsley need to be mixed into the peppers and mushrooms before serving.\"\n}", "relevant_step_nos": [9, 10]}
{"plan_idx": 35, "title": "peppers-with-tuna-and-mushrooms", "question_idx": 17, "steps": ["Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.", "Cook for 5 minutes or until softened.", "Add 250g halved or quartered chestnut mushrooms.", "Season and cook, stirring occasionally, until the mushrooms give up their juices.", "Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.", "Cook, stirring until the mushrooms are evenly coated with a little sauce.", "Meanwhile, drain and slice a 280g jar of roasted red peppers.", "Mix the peppers into the mushrooms and heat gently for 1 minute.", "Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.", "Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 10?", "why_question": "Why must Step 5 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: peppers-with-tuna-and-mushrooms\n\nProcedure:\n1. Heat 2 tbsp olive oil in a frying pan and add 1 chopped onion.\n2. Cook for 5 minutes or until softened.\n3. Add 250g halved or quartered chestnut mushrooms.\n4. Season and cook, stirring occasionally, until the mushrooms give up their juices.\n5. Add 1 tbsp wholegrain mustard, the grated zest of 1 lemon, 1 crushed garlic clove and 4 tbsp dry sherry.\n6. Cook, stirring until the mushrooms are evenly coated with a little sauce.\n7. Meanwhile, drain and slice a 280g jar of roasted red peppers.\n8. Mix the peppers into the mushrooms and heat gently for 1 minute.\n9. Finally, stir in a 185g can tuna, drained, and 25g chopped fresh parsley.\n10. Serve immediately, with crisp salad leaves and plenty of hot crusty bread or boiled rice.\n\n\n1. Must Step 5 happen before Step 10?\n2. Why must Step 5 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 10 because the sauce needs to be evenly coated on the mushrooms before adding the tuna and parsley.\" \n}", "relevant_step_nos": [5, 10]}
{"plan_idx": 36, "title": "rouille", "question_idx": 0, "steps": ["In a mortar, crush the garlic to a fine mash.", "Transfer to a mixing bowl.", "Add the egg yolks, and beat into a mayonnaise while adding the olive oil.", "When the sauce is thick and smooth, stir in the cayenne pepper and saffron."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: rouille\n\nProcedure:\n1. In a mortar, crush the garlic to a fine mash.\n2. Transfer to a mixing bowl.\n3. Add the egg yolks, and beat into a mayonnaise while adding the olive oil.\n4. When the sauce is thick and smooth, stir in the cayenne pepper and saffron.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because the egg yolks and olive oil need to be beaten into a mayonnaise before adding the cayenne pepper and saffron.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 36, "title": "rouille", "question_idx": 1, "steps": ["In a mortar, crush the garlic to a fine mash.", "Transfer to a mixing bowl.", "Add the egg yolks, and beat into a mayonnaise while adding the olive oil.", "When the sauce is thick and smooth, stir in the cayenne pepper and saffron."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 4?", "why_question": "Why must Step 1 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: rouille\n\nProcedure:\n1. In a mortar, crush the garlic to a fine mash.\n2. Transfer to a mixing bowl.\n3. Add the egg yolks, and beat into a mayonnaise while adding the olive oil.\n4. When the sauce is thick and smooth, stir in the cayenne pepper and saffron.\n\n\n1. Must Step 1 happen before Step 4?\n2. Why must Step 1 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 4 because the crushed garlic needs to be incorporated into the sauce before adding the cayenne pepper and saffron.\"\n}", "relevant_step_nos": [1, 4]}
{"plan_idx": 36, "title": "rouille", "question_idx": 2, "steps": ["In a mortar, crush the garlic to a fine mash.", "Transfer to a mixing bowl.", "Add the egg yolks, and beat into a mayonnaise while adding the olive oil.", "When the sauce is thick and smooth, stir in the cayenne pepper and saffron."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: rouille\n\nProcedure:\n1. In a mortar, crush the garlic to a fine mash.\n2. Transfer to a mixing bowl.\n3. Add the egg yolks, and beat into a mayonnaise while adding the olive oil.\n4. When the sauce is thick and smooth, stir in the cayenne pepper and saffron.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to prepare the base for the mayonnaise before adding the olive oil in Step 3.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 37, "title": "sardine spread", "question_idx": 0, "steps": ["Place the sardines in a bowl and roughly mash with a fork.", "Add in all of the remaining ingredients and stir well.", "Enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: sardine spread\n\nProcedure:\n1. Place the sardines in a bowl and roughly mash with a fork.\n2. Add in all of the remaining ingredients and stir well.\n3. Enjoy!\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 2 because the sardines need to be mashed before adding the remaining ingredients.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 38, "title": "scone-pastry", "question_idx": 0, "steps": ["Sift the flour, baking powder and salt into a bowl.", "Add the butter and rub in until the mixture resembles fine crumbs.", "Make a well in the centre and add the milk.", "Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.", "Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: scone-pastry\n\nProcedure:\n1. Sift the flour, baking powder and salt into a bowl.\n2. Add the butter and rub in until the mixture resembles fine crumbs.\n3. Make a well in the centre and add the milk.\n4. Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.\n5. Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 because the dough needs to be mixed with milk before it is kneaded and rolled out.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 38, "title": "scone-pastry", "question_idx": 1, "steps": ["Sift the flour, baking powder and salt into a bowl.", "Add the butter and rub in until the mixture resembles fine crumbs.", "Make a well in the centre and add the milk.", "Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.", "Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: scone-pastry\n\nProcedure:\n1. Sift the flour, baking powder and salt into a bowl.\n2. Add the butter and rub in until the mixture resembles fine crumbs.\n3. Make a well in the centre and add the milk.\n4. Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.\n5. Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the dough needs to be at the right consistency before it can be kneaded and rolled out.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 38, "title": "scone-pastry", "question_idx": 2, "steps": ["Sift the flour, baking powder and salt into a bowl.", "Add the butter and rub in until the mixture resembles fine crumbs.", "Make a well in the centre and add the milk.", "Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.", "Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: scone-pastry\n\nProcedure:\n1. Sift the flour, baking powder and salt into a bowl.\n2. Add the butter and rub in until the mixture resembles fine crumbs.\n3. Make a well in the centre and add the milk.\n4. Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.\n5. Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required.\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 sifts the dry ingredients before adding the butter, ensuring even distribution and a consistent texture in the dough.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 38, "title": "scone-pastry", "question_idx": 3, "steps": ["Sift the flour, baking powder and salt into a bowl.", "Add the butter and rub in until the mixture resembles fine crumbs.", "Make a well in the centre and add the milk.", "Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.", "Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: scone-pastry\n\nProcedure:\n1. Sift the flour, baking powder and salt into a bowl.\n2. Add the butter and rub in until the mixture resembles fine crumbs.\n3. Make a well in the centre and add the milk.\n4. Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.\n5. Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 3 because the butter needs to be rubbed into the dry ingredients before adding the milk to create the right texture for the dough.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 38, "title": "scone-pastry", "question_idx": 4, "steps": ["Sift the flour, baking powder and salt into a bowl.", "Add the butter and rub in until the mixture resembles fine crumbs.", "Make a well in the centre and add the milk.", "Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.", "Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: scone-pastry\n\nProcedure:\n1. Sift the flour, baking powder and salt into a bowl.\n2. Add the butter and rub in until the mixture resembles fine crumbs.\n3. Make a well in the centre and add the milk.\n4. Mix with a wooden spoon or your fingers to make a fairly soft but not sticky dough, adding a little more milk if necessary.\n5. Transfer the dough to a lightly floured surface and knead briefly, then roll out and use as required.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because rubbing in the butter is necessary before kneading and rolling out the dough.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 0, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 because the corned beef needs to simmer with the vegetables before adding the gravy granules to thicken the stew.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 1, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because the potatoes, carrots, and onion need to be cooked before the stew is ready to serve.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 2, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because simmering the ingredients for 20 minutes allows the stew to thicken up when the gravy granules are added in Step 4.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 3, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 thickens the stew, making it ready to serve in Step 6.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 4, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 5?", "why_question": "Why must Step 1 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 1 happen before Step 5?\n2. Why must Step 1 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to cook the potatoes, carrots, and onion before adding the gravy granules in Step 4 to thicken the stew.\"\n}", "relevant_step_nos": [1, 5]}
{"plan_idx": 39, "title": "shellabina-s-corned-beef-hash", "question_idx": 5, "steps": ["Add potatoes, carrots and onion to a large pan and add water to cover.", "Bring to boil and simmer for 20 minutes.", "Add corned beef and simmer for another 5 minutes.", "Add gravy granules and stir.", "The stew will now thicken up.", "It is now ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 6?", "why_question": "Why must Step 1 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: shellabina-s-corned-beef-hash\n\nProcedure:\n1. Add potatoes, carrots and onion to a large pan and add water to cover.\n2. Bring to boil and simmer for 20 minutes.\n3. Add corned beef and simmer for another 5 minutes.\n4. Add gravy granules and stir.\n5. The stew will now thicken up.\n6. It is now ready to serve.\n\n\n1. Must Step 1 happen before Step 6?\n2. Why must Step 1 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to cook the potatoes, carrots, and onion before serving the dish.\"\n}", "relevant_step_nos": [1, 6]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 0, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 8 must happen before Step 4 because the cake needs to be fully baked before checking if it's done with a skewer.\"\n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 1, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 10?", "why_question": "Why must Step 1 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 1 happen before Step 10?\n2. Why must Step 1 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 10 because the oven needs to be preheated before baking the cupcakes.\"\n}", "relevant_step_nos": [1, 10]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 2, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 9?", "why_question": "Why must Step 5 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 5 happen before Step 9?\n2. Why must Step 5 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 9 because the cupcakes need to be fully baked before allowing them to cool.\"\n}", "relevant_step_nos": [5, 9]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 3, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 8 because the flour needs to be mixed well into the batter before baking to ensure the cupcakes are evenly cooked.\"\n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 4, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 10?", "why_question": "Why must Step 7 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 7 happen before Step 10?\n2. Why must Step 7 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 10 because the cupcakes need to be baked before they can be decorated.\"\n}", "relevant_step_nos": [7, 10]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 5, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 9?", "why_question": "Why must Step 2 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 2 happen before Step 9?\n2. Why must Step 2 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 9 because beating the butter and sugar together is part of the initial mixing process, which needs to be done before allowing the cupcakes to cool fully.\"\n}", "relevant_step_nos": [2, 9]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 6, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 because the eggs need to be fully incorporated into the batter before adding the flour.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 7, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 6 because the eggs need to be fully incorporated into the batter before spooning it into the muffin tin.\"\n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 40, "title": "simple-basic-cupcakes", "question_idx": 8, "steps": ["Preheat oven to 180 C / Gas 4.", "Beat butter and sugar together until light and fluffy.", "Add the eggs, one at a time, beating after each addition.", "If the batter is too thick, stir in some milk.", "Gradually add the flour, mixing well each time until evenly mixed.", "Spoon tablespoons of batter into a greased muffin tin.", "Bake in preheated oven for 15-20 minutes.", "The cake is done when a skewer inserted to the cake comes out clean.", "Let rest for fives minutes, then allow to cool fully on a wire rack.", "Decorate, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: simple-basic-cupcakes\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Beat butter and sugar together until light and fluffy.\n3. Add the eggs, one at a time, beating after each addition.\n4. If the batter is too thick, stir in some milk.\n5. Gradually add the flour, mixing well each time until evenly mixed.\n6. Spoon tablespoons of batter into a greased muffin tin.\n7. Bake in preheated oven for 15-20 minutes.\n8. The cake is done when a skewer inserted to the cake comes out clean.\n9. Let rest for fives minutes, then allow to cool fully on a wire rack.\n10. Decorate, if desired.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the batter needs to be spooned into the muffin tin before baking.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 0, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Seasoning the dish with salt before cooking allows the flavors to develop and meld together during the cooking process.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 1, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 8?", "why_question": "Why must Step 7 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 7 happen before Step 8?\n2. Why must Step 7 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 8 because the chicken needs to be cooked for 4 to 6 hours on High or 6 to 8 hours on Low before removing the cardamom pods and serving.\" \n}", "relevant_step_nos": [7, 8]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 2, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the ingredients need to be mixed and combined before transferring to the slow cooker.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 3, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 4?", "why_question": "Why must Step 1 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 1 happen before Step 4?\n2. Why must Step 1 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to properly cook the chicken, onion, and garlic before adding the spices in Step 4.\"\n}", "relevant_step_nos": [1, 4]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 4, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 7?", "why_question": "Why must Step 1 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 1 happen before Step 7?\n2. Why must Step 1 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to cook the chicken before slow cooking it in the slow cooker in Step 7.\"\n}", "relevant_step_nos": [1, 7]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 5, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to heat the butter and oil before adding the chicken, onion, and garlic.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 6, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 6?", "why_question": "Why must Step 1 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 1 happen before Step 6?\n2. Why must Step 1 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 6 because the ingredients need to be cooked and combined before being transferred to the slow cooker for further cooking.\"\n}", "relevant_step_nos": [1, 6]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 7, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 is necessary to cook the chicken and onion before transferring to the slow cooker for the final cooking process.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 8, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 because the ingredients need to be cooked and combined before transferring to the slow cooker.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 9, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because the onion needs to soften and turn translucent before adding the curry powder, curry paste, tandoori masala, garam masala, and tomato puree.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 10, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 7 because the ingredients need to be combined and cooked in the slow cooker for the specified time to achieve the desired flavor and consistency.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 11, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the ingredients need to be combined and seasoned before cooking in the slow cooker.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 41, "title": "slow cooker butter chicken", "question_idx": 12, "steps": ["Heat the butter and vegetable oil in a large frying pan over medium heat.", "Stir in the chicken, onion and garlic.", "Cook and stir until the onion has softened and turned translucent, about 10 minutes.", "Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.", "Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.", "Season to taste with salt.", "Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.", "Remove and discard the cardamom pods before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: slow cooker butter chicken\n\nProcedure:\n1. Heat the butter and vegetable oil in a large frying pan over medium heat.\n2. Stir in the chicken, onion and garlic.\n3. Cook and stir until the onion has softened and turned translucent, about 10 minutes.\n4. Stir in the curry powder, curry paste, tandoori masala, garam masala and tomato puree until no lumps of tomato puree remain.\n5. Pour into a slow cooker, and stir in the cardamom pods, coconut milk and yoghurt.\n6. Season to taste with salt.\n7. Cook on High 4 to 6 hours, or on Low 6 to 8 hours until the chicken is tender and the sauce has reduced to your desired consistency.\n8. Remove and discard the cardamom pods before serving.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because the ingredients need to be cooked and combined before transferring to the slow cooker.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 0, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 8 because the salad needs to be arranged with grapefruit segments and smoked salmon before the dressing is poured over it.\"\n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 1, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 8?", "why_question": "Why must Step 2 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 2 happen before Step 8?\n2. Why must Step 2 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to collect the grapefruit juice for the dressing in Step 8.\"\n}", "relevant_step_nos": [2, 8]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 2, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 7?", "why_question": "Why must Step 2 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 2 happen before Step 7?\n2. Why must Step 2 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 7 because the grapefruit juice from the bowl is needed to mix with cracked peppercorns in Step 7.\"\n}", "relevant_step_nos": [2, 7]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 3, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because the grapefruit segments and juice are needed to be arranged and used in the salad.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 4, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 7?", "why_question": "Why must Step 1 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 1 happen before Step 7?\n2. Why must Step 1 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 7 because the grapefruits need to be prepared and the juice collected before mixing it with peppercorns for the salad dressing.\"\n}", "relevant_step_nos": [1, 7]}
{"plan_idx": 42, "title": "smoked-salmon-and-grapefruit-salad", "question_idx": 5, "steps": ["Top and tail the grapefruits.", "Slice the skin off the grapefruit over a bowl.", "This is to catch any juices.", "Using a small knife, segment the grapefruits over the juice bowl.", "Arrange the lamb's lettuce onto each serving plate or a large salad bowl.", "Top with grapefruit segments and smoked salmon.", "Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.", "Pour over the salad and serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: smoked-salmon-and-grapefruit-salad\n\nProcedure:\n1. Top and tail the grapefruits.\n2. Slice the skin off the grapefruit over a bowl.\n3. This is to catch any juices.\n4. Using a small knife, segment the grapefruits over the juice bowl.\n5. Arrange the lamb's lettuce onto each serving plate or a large salad bowl.\n6. Top with grapefruit segments and smoked salmon.\n7. Mix together 1 pinch of cracked peppercorns and 1 tablespoon grapefruit juice from the bowl.\n8. Pour over the salad and serve.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 8 because the grapefruit juice from segmenting the grapefruits is used to mix with cracked peppercorns for the salad dressing in Step 7.\"\n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 43, "title": "spanish-rabbit-stew", "question_idx": 0, "steps": ["After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?", ").", "I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.", "Pour in half pint of white wine and half pint of water.", "You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.", "Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.", "Turn your slow cooker on to low power, let it simmer for about 5 hours.", "If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!", "Flavour with salt and pepper to taste.", "Cook the pasta according to the packet instructions, and serve with pasta.", "Most importantly, enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 10?", "why_question": "Why must Step 4 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spanish-rabbit-stew\n\nProcedure:\n1. After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?\n2. ).\n3. I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.\n4. Pour in half pint of white wine and half pint of water.\n5. You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.\n6. Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.\n7. Turn your slow cooker on to low power, let it simmer for about 5 hours.\n8. If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!\n9. Flavour with salt and pepper to taste.\n10. Cook the pasta according to the packet instructions, and serve with pasta.\n11. Most importantly, enjoy!\n\n\n1. Must Step 4 happen before Step 10?\n2. Why must Step 4 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 10 can happen independently of Step 4 as it involves cooking the pasta separately and serving it with the stew.\"\n}", "relevant_step_nos": [4, 10]}
{"plan_idx": 43, "title": "spanish-rabbit-stew", "question_idx": 1, "steps": ["After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?", ").", "I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.", "Pour in half pint of white wine and half pint of water.", "You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.", "Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.", "Turn your slow cooker on to low power, let it simmer for about 5 hours.", "If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!", "Flavour with salt and pepper to taste.", "Cook the pasta according to the packet instructions, and serve with pasta.", "Most importantly, enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 10?", "why_question": "Why must Step 1 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spanish-rabbit-stew\n\nProcedure:\n1. After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?\n2. ).\n3. I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.\n4. Pour in half pint of white wine and half pint of water.\n5. You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.\n6. Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.\n7. Turn your slow cooker on to low power, let it simmer for about 5 hours.\n8. If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!\n9. Flavour with salt and pepper to taste.\n10. Cook the pasta according to the packet instructions, and serve with pasta.\n11. Most importantly, enjoy!\n\n\n1. Must Step 1 happen before Step 10?\n2. Why must Step 1 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to prepare the ingredients for the stew before cooking the pasta.\"\n}", "relevant_step_nos": [1, 10]}
{"plan_idx": 43, "title": "spanish-rabbit-stew", "question_idx": 2, "steps": ["After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?", ").", "I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.", "Pour in half pint of white wine and half pint of water.", "You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.", "Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.", "Turn your slow cooker on to low power, let it simmer for about 5 hours.", "If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!", "Flavour with salt and pepper to taste.", "Cook the pasta according to the packet instructions, and serve with pasta.", "Most importantly, enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 4?", "why_question": "Why must Step 1 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spanish-rabbit-stew\n\nProcedure:\n1. After slicing all of your ingredients, throw them all into your slow cooker (everything in at once, couldn\u2019t be easier right?\n2. ).\n3. I like to mix mine up, so I can take a big scoop and get a lovely mix of goodness.\n4. Pour in half pint of white wine and half pint of water.\n5. You might want to add a little more of each to cover your ingredients but the key rule is to keep the mixture 50/50.\n6. Make sure your tarragon and bay leaf aren\u2019t floating on the top or it won't let off any real flavour.\n7. Turn your slow cooker on to low power, let it simmer for about 5 hours.\n8. If your aren\u2019t hungry once that wonderful smell fills the room, I\u2019ll refund the cost of the ingredients!\n9. Flavour with salt and pepper to taste.\n10. Cook the pasta according to the packet instructions, and serve with pasta.\n11. Most importantly, enjoy!\n\n\n1. Must Step 1 happen before Step 4?\n2. Why must Step 1 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 4 because the ingredients need to be in the slow cooker before adding the white wine and water.\"\n}", "relevant_step_nos": [1, 4]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 0, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 15?", "why_question": "Why must Step 2 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 2 happen before Step 15?\n2. Why must Step 2 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 2 does not need to happen before Step 15 as they are independent steps in the procedure.\"\n}", "relevant_step_nos": [2, 15]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 1, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 17?", "why_question": "Why must Step 2 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 2 happen before Step 17?\n2. Why must Step 2 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to prepare the squash for baking, which is a key ingredient in the soup.\"\n}", "relevant_step_nos": [2, 17]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 2, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to create steam and prevent the squash from drying out while baking in the oven.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 3, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because the squash needs to be baked before it can be peeled and added to the soup.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 4, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 12?", "why_question": "Why must Step 2 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 2 happen before Step 12?\n2. Why must Step 2 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to bake the squash before mashing it with the potatoes in Step 12.\"\n}", "relevant_step_nos": [2, 12]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 5, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 15?", "why_question": "Why must Step 4 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 4 happen before Step 15?\n2. Why must Step 4 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 15 because the squash needs to be cooked and mashed before seasoning and adding the other ingredients.\"\n}", "relevant_step_nos": [4, 15]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 6, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 16?", "why_question": "Why must Step 4 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 4 happen before Step 16?\n2. Why must Step 4 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 16 because the squash needs to be baked and then added to the soup before heating through.\"\n}", "relevant_step_nos": [4, 16]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 7, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the squash needs to be baked before the peel can be easily removed.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 8, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 13?", "why_question": "Why must Step 4 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 4 happen before Step 13?\n2. Why must Step 4 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 13 because the squash needs to be baked and cooled before it can be pureed in Step 13.\"\n}", "relevant_step_nos": [4, 13]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 9, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 14?", "why_question": "Why must Step 4 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 4 happen before Step 14?\n2. Why must Step 4 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 14 because the squash needs to be baked and then added to the soup before seasoning and heating through.\"\n}", "relevant_step_nos": [4, 14]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 10, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 17?", "why_question": "Why must Step 4 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 4 happen before Step 17?\n2. Why must Step 4 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 17 because the squash needs to be baked and then added to the soup before serving.\"\n}", "relevant_step_nos": [4, 17]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 11, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 12?", "why_question": "Why must Step 4 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 4 happen before Step 12?\n2. Why must Step 4 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 12 because the squash needs to be baked and softened before it can be mashed with the potatoes in Step 12.\"\n}", "relevant_step_nos": [4, 12]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 12, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 13?", "why_question": "Why must Step 7 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 7 happen before Step 13?\n2. Why must Step 7 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 13 because the squash needs to be added to the pot before it can be pureed.\"\n}", "relevant_step_nos": [7, 13]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 13, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 15?", "why_question": "Why must Step 7 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 7 happen before Step 15?\n2. Why must Step 7 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 15 because the butter needs to be melted before the spices and other ingredients are added and heated through.\"\n}", "relevant_step_nos": [7, 15]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 14, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 9?", "why_question": "Why must Step 7 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 7 happen before Step 9?\n2. Why must Step 7 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 9 because the onions, leek, and garlic need to be saut\u00e9ed before adding the chicken stock.\"\n}", "relevant_step_nos": [7, 9]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 15, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 16?", "why_question": "Why must Step 7 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 7 happen before Step 16?\n2. Why must Step 7 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 16 because the soup needs to be heated through before ladling into bowls.\"\n}", "relevant_step_nos": [7, 16]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 16, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 11?", "why_question": "Why must Step 7 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 7 happen before Step 11?\n2. Why must Step 7 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 11 because the onions, leek, and garlic need to be saut\u00e9ed before adding the potatoes and bringing to a boil.\"\n}", "relevant_step_nos": [7, 11]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 17, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 16?", "why_question": "Why must Step 3 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 3 happen before Step 16?\n2. Why must Step 3 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 16 because the squash needs to be baked and cooled before being added to the soup.\" \n}", "relevant_step_nos": [3, 16]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 18, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because the squash needs to be baked with the cut side down to cook properly and become tender.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 19, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 12?", "why_question": "Why must Step 3 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 3 happen before Step 12?\n2. Why must Step 3 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 12 because the squash needs to be baked and then cooled before it can be added and mashed with the potatoes in the soup.\"\n}", "relevant_step_nos": [3, 12]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 20, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 15?", "why_question": "Why must Step 3 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 3 happen before Step 15?\n2. Why must Step 3 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 15 because the squash needs to be baked and peeled before it can be seasoned and added to the soup.\"\n}", "relevant_step_nos": [3, 15]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 21, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 11?", "why_question": "Why must Step 9 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 9 happen before Step 11?\n2. Why must Step 9 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 11 because the potatoes need to cook in the chicken stock before adding the squash.\"\n}", "relevant_step_nos": [9, 11]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 22, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 14?", "why_question": "Why must Step 9 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 9 happen before Step 14?\n2. Why must Step 9 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 14 because the chicken stock needs to be added and the potatoes need to be cooked before the soup is seasoned and the other ingredients are added.\"\n}", "relevant_step_nos": [9, 14]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 23, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 12?", "why_question": "Why must Step 11 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 11 happen before Step 12?\n2. Why must Step 11 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 12 because the potatoes need to be cooked and softened before adding the squash and mashing them together.\"\n}", "relevant_step_nos": [11, 12]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 24, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 16?", "why_question": "Why must Step 11 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 11 happen before Step 16?\n2. Why must Step 11 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 16 because the soup needs to be cooked until the potatoes are soft before heating through, but not boiling, in Step 16.\"\n}", "relevant_step_nos": [11, 16]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 25, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 14?", "why_question": "Why must Step 11 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 11 happen before Step 14?\n2. Why must Step 11 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 14 because the squash needs to be cooked and mashed with the potatoes before seasoning and heating the soup.\"\n}", "relevant_step_nos": [11, 14]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 26, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 17?", "why_question": "Why must Step 11 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 11 happen before Step 17?\n2. Why must Step 11 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 17 because the soup needs to be cooked until soft before ladling into bowls.\"\n}", "relevant_step_nos": [11, 17]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 27, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 14 happen before Step 15?", "why_question": "Why must Step 14 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 14 happen before Step 15?\n2. Why must Step 14 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 15 can happen before Step 14 because the seasoning and adding of sherry, cream, and milk can be done in any order.\"\n}", "relevant_step_nos": [14, 15]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 28, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 14 happen before Step 17?", "why_question": "Why must Step 14 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 14 happen before Step 17?\n2. Why must Step 14 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen before Step 17 because the soup needs to be seasoned before it is ladled into bowls.\"\n}", "relevant_step_nos": [14, 17]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 29, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 14?", "why_question": "Why must Step 10 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 10 happen before Step 14?\n2. Why must Step 10 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 14 because the squash needs to be cooked and mashed with the potatoes before seasoning and blending the soup.\"\n}", "relevant_step_nos": [10, 14]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 30, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 15?", "why_question": "Why must Step 12 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 12 happen before Step 15?\n2. Why must Step 12 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen before Step 15 because the spices and seasonings need to be added after the squash is mashed and pureed to ensure they are evenly distributed throughout the soup.\"\n}", "relevant_step_nos": [12, 15]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 31, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 16?", "why_question": "Why must Step 12 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 12 happen before Step 16?\n2. Why must Step 12 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen before Step 16 because the soup needs to be pureed before being heated through.\"\n}", "relevant_step_nos": [12, 16]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 32, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the squash needs to cool slightly before removing the peel.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 33, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 16 happen before Step 17?", "why_question": "Why must Step 16 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 16 happen before Step 17?\n2. Why must Step 16 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 16 must happen before Step 17 because the soup needs to be heated through before ladling into bowls.\"\n}", "relevant_step_nos": [16, 17]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 34, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 12?", "why_question": "Why must Step 1 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 1 happen before Step 12?\n2. Why must Step 1 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 preheats the oven to bake the squash, which is needed before mashing it in Step 12.\"\n}", "relevant_step_nos": [1, 12]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 35, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 15?", "why_question": "Why must Step 1 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 1 happen before Step 15?\n2. Why must Step 1 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 15 because the oven needs to be preheated before the soup is seasoned and heated through.\"\n}", "relevant_step_nos": [1, 15]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 36, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 13?", "why_question": "Why must Step 1 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 1 happen before Step 13?\n2. Why must Step 1 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 13 because the squash needs to be baked and cooled before it can be pureed in Step 13.\"\n}", "relevant_step_nos": [1, 13]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 37, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 17?", "why_question": "Why must Step 13 happen before Step 17?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 13 happen before Step 17?\n2. Why must Step 13 happen before Step 17?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 17 because the soup needs to be pureed and smooth before ladling into bowls.\"\n}", "relevant_step_nos": [13, 17]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 38, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 15?", "why_question": "Why must Step 13 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 13 happen before Step 15?\n2. Why must Step 13 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 15 because the spices and other seasonings need to be blended into the soup before adding the sherry, cream, and milk.\"\n}", "relevant_step_nos": [13, 15]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 39, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 13 happen before Step 16?", "why_question": "Why must Step 13 happen before Step 16?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 13 happen before Step 16?\n2. Why must Step 13 happen before Step 16?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 13 must happen before Step 16 because the soup needs to be pureed before it is heated through.\"\n}", "relevant_step_nos": [13, 16]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 40, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 14?", "why_question": "Why must Step 8 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 8 happen before Step 14?\n2. Why must Step 8 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 14 because the sauteed onion, leek, and garlic need to be seasoned and blended with the soup before it is seasoned with spices and other ingredients.\"\n}", "relevant_step_nos": [8, 14]}
{"plan_idx": 44, "title": "spiced butternut squash soup", "question_idx": 41, "steps": ["Preheat the oven to 190 C / Gas mark 6.", "Pour a small layer of water in a baking dish, or a swiss roll tin.", "Place the squash halves cut side down on the dish.", "Bake for about 40 minutes, or until a fork can easily pierce the flesh.", "Cool slightly, then remove the peel.", "Set aside.", "Melt the butter in a large pot over medium heat.", "Add the onion, leek and garlic, and saute for a few minutes, until tender.", "Pour the chicken stock into the pot.", "Add the potatoes, and bring to the boil.", "Cook for about 20 minutes, or until soft.", "Add the squash, and mash with the potatoes until chunks are small.", "Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.", "Return to the pot.", "Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.", "Heat through, but do not boil.", "Ladle into bowls, and top with a dollop of soured cream if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced butternut squash soup\n\nProcedure:\n1. Preheat the oven to 190 C / Gas mark 6.\n2. Pour a small layer of water in a baking dish, or a swiss roll tin.\n3. Place the squash halves cut side down on the dish.\n4. Bake for about 40 minutes, or until a fork can easily pierce the flesh.\n5. Cool slightly, then remove the peel.\n6. Set aside.\n7. Melt the butter in a large pot over medium heat.\n8. Add the onion, leek and garlic, and saute for a few minutes, until tender.\n9. Pour the chicken stock into the pot.\n10. Add the potatoes, and bring to the boil.\n11. Cook for about 20 minutes, or until soft.\n12. Add the squash, and mash with the potatoes until chunks are small.\n13. Use an immersion hand blender to puree the soup, or transfer to a blender or food processor in batches, and puree until smooth.\n14. Return to the pot.\n15. Season the soup with cayenne pepper, allspice, nutmeg, ginger, salt and pepper, then stir in the sherry, cream and milk.\n16. Heat through, but do not boil.\n17. Ladle into bowls, and top with a dollop of soured cream if desired.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 9 because the onion, leek, and garlic need to be saut\u00e9ed before adding the chicken stock.\"\n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 45, "title": "spiced-guava-cake", "question_idx": 0, "steps": ["Preheat oven to 180 C / Gas 4.", "Grease and flour one 23x13cm loaf tin.", "Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.", "In another bowl, mix together guava pulp and juice.", "In another bowl, cream together butter and sugar.", "Add eggs, one at a time.", "Add flour mixture and guava mixture alternately to creamed mixture.", "Pour batter into prepared tin.", "Bake at 180 C / Gas 4 for 30-35 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 9?", "why_question": "Why must Step 7 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-guava-cake\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Grease and flour one 23x13cm loaf tin.\n3. Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.\n4. In another bowl, mix together guava pulp and juice.\n5. In another bowl, cream together butter and sugar.\n6. Add eggs, one at a time.\n7. Add flour mixture and guava mixture alternately to creamed mixture.\n8. Pour batter into prepared tin.\n9. Bake at 180 C / Gas 4 for 30-35 minutes.\n\n\n1. Must Step 7 happen before Step 9?\n2. Why must Step 7 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 9 to ensure that the batter is properly mixed before baking.\"\n}", "relevant_step_nos": [7, 9]}
{"plan_idx": 45, "title": "spiced-guava-cake", "question_idx": 1, "steps": ["Preheat oven to 180 C / Gas 4.", "Grease and flour one 23x13cm loaf tin.", "Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.", "In another bowl, mix together guava pulp and juice.", "In another bowl, cream together butter and sugar.", "Add eggs, one at a time.", "Add flour mixture and guava mixture alternately to creamed mixture.", "Pour batter into prepared tin.", "Bake at 180 C / Gas 4 for 30-35 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-guava-cake\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Grease and flour one 23x13cm loaf tin.\n3. Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.\n4. In another bowl, mix together guava pulp and juice.\n5. In another bowl, cream together butter and sugar.\n6. Add eggs, one at a time.\n7. Add flour mixture and guava mixture alternately to creamed mixture.\n8. Pour batter into prepared tin.\n9. Bake at 180 C / Gas 4 for 30-35 minutes.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the eggs are fully incorporated into the mixture before adding the flour and guava mixture.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 45, "title": "spiced-guava-cake", "question_idx": 2, "steps": ["Preheat oven to 180 C / Gas 4.", "Grease and flour one 23x13cm loaf tin.", "Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.", "In another bowl, mix together guava pulp and juice.", "In another bowl, cream together butter and sugar.", "Add eggs, one at a time.", "Add flour mixture and guava mixture alternately to creamed mixture.", "Pour batter into prepared tin.", "Bake at 180 C / Gas 4 for 30-35 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-guava-cake\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Grease and flour one 23x13cm loaf tin.\n3. Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.\n4. In another bowl, mix together guava pulp and juice.\n5. In another bowl, cream together butter and sugar.\n6. Add eggs, one at a time.\n7. Add flour mixture and guava mixture alternately to creamed mixture.\n8. Pour batter into prepared tin.\n9. Bake at 180 C / Gas 4 for 30-35 minutes.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the creaming of butter and sugar is a standard baking technique that should be done before adding eggs.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 45, "title": "spiced-guava-cake", "question_idx": 3, "steps": ["Preheat oven to 180 C / Gas 4.", "Grease and flour one 23x13cm loaf tin.", "Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.", "In another bowl, mix together guava pulp and juice.", "In another bowl, cream together butter and sugar.", "Add eggs, one at a time.", "Add flour mixture and guava mixture alternately to creamed mixture.", "Pour batter into prepared tin.", "Bake at 180 C / Gas 4 for 30-35 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-guava-cake\n\nProcedure:\n1. Preheat oven to 180 C / Gas 4.\n2. Grease and flour one 23x13cm loaf tin.\n3. Sieve together flour, cornflour, nutmeg, bicarbonate of soda, cinnamon and cloves.\n4. In another bowl, mix together guava pulp and juice.\n5. In another bowl, cream together butter and sugar.\n6. Add eggs, one at a time.\n7. Add flour mixture and guava mixture alternately to creamed mixture.\n8. Pour batter into prepared tin.\n9. Bake at 180 C / Gas 4 for 30-35 minutes.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because the guava mixture needs to be prepared before it can be added to the batter.\" \n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 0, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 13?", "why_question": "Why must Step 9 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 9 happen before Step 13?\n2. Why must Step 9 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 13 because the teabread needs to be turned out onto a wire rack to cool before it can be brushed with honey.\"\n}", "relevant_step_nos": [9, 13]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 1, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 10?", "why_question": "Why must Step 9 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 9 happen before Step 10?\n2. Why must Step 9 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 10 because the mixture needs to be smoothed out before sprinkling with pumpkin seeds.\"\n}", "relevant_step_nos": [9, 10]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 2, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 11?", "why_question": "Why must Step 9 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 9 happen before Step 11?\n2. Why must Step 9 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 11 because the mixture needs to be placed in the tin and smoothed before baking.\"\n}", "relevant_step_nos": [9, 11]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 3, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 12?", "why_question": "Why must Step 9 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 9 happen before Step 12?\n2. Why must Step 9 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 12 because the teabread needs to cool in the tin for 10 minutes before being turned out onto a wire rack.\"\n}", "relevant_step_nos": [9, 12]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 4, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 13?", "why_question": "Why must Step 5 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 5 happen before Step 13?\n2. Why must Step 5 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 13 because the teabread needs to be brushed with honey while still warm, and the honey is added in Step 5.\"\n}", "relevant_step_nos": [5, 13]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 5, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 9?", "why_question": "Why must Step 5 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 5 happen before Step 9?\n2. Why must Step 5 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 9 because the pumpkin mixture needs to be added and evenly mixed before the mixture is tipped into the prepared tin and smoothed.\" \n}", "relevant_step_nos": [5, 9]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 6, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 14?", "why_question": "Why must Step 5 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 5 happen before Step 14?\n2. Why must Step 5 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 14 because the pumpkin mixture needs to be prepared and added to the teabread before it is brushed with honey.\"\n}", "relevant_step_nos": [5, 14]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 7, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 12?", "why_question": "Why must Step 5 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 5 happen before Step 12?\n2. Why must Step 5 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 involves adding ingredients to the mixture, which must be done before baking in Step 12.\"\n}", "relevant_step_nos": [5, 12]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 8, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 10?", "why_question": "Why must Step 5 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 5 happen before Step 10?\n2. Why must Step 5 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 10 because the pumpkin seeds need to be sprinkled on the mixture before baking.\"\n}", "relevant_step_nos": [5, 10]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 9, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 13?", "why_question": "Why must Step 11 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 11 happen before Step 13?\n2. Why must Step 11 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 13 because the teabread needs to be baked and cooled before being brushed with honey.\"\n}", "relevant_step_nos": [11, 13]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 10, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 12?", "why_question": "Why must Step 11 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 11 happen before Step 12?\n2. Why must Step 11 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the loaf to cool slightly before removing it from the tin.\"\n}", "relevant_step_nos": [11, 12]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 11, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 9?", "why_question": "Why must Step 2 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 2 happen before Step 9?\n2. Why must Step 2 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to prepare the tin for the mixture to be poured into it in Step 9.\"\n}", "relevant_step_nos": [2, 9]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 12, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 13?", "why_question": "Why must Step 2 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 2 happen before Step 13?\n2. Why must Step 2 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to prepare the tin for the teabread before brushing it with honey in Step 13.\"\n}", "relevant_step_nos": [2, 13]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 13, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the flours and allspice need to be sifted and combined before rubbing in the butter.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 14, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 11?", "why_question": "Why must Step 6 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 6 happen before Step 11?\n2. Why must Step 6 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 11 because the mixture needs to be prepared and put into the tin before baking.\"\n}", "relevant_step_nos": [6, 11]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 15, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 12?", "why_question": "Why must Step 6 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 6 happen before Step 12?\n2. Why must Step 6 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 12 because the mixture needs to be prepared and put into the tin before baking.\" \n}", "relevant_step_nos": [6, 12]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 16, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 13?", "why_question": "Why must Step 6 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 6 happen before Step 13?\n2. Why must Step 6 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 13 because the mixture needs to be prepared and baked before being brushed with honey.\"\n}", "relevant_step_nos": [6, 13]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 17, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 14?", "why_question": "Why must Step 3 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 3 happen before Step 14?\n2. Why must Step 3 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 14 because the pumpkin flesh needs to be cooked and mashed before the teabread is made.\"\n}", "relevant_step_nos": [3, 14]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 18, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 8?", "why_question": "Why must Step 3 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 3 happen before Step 8?\n2. Why must Step 3 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 8 because the pumpkin needs to be cooked and mashed before being added to the flour mixture.\"\n}", "relevant_step_nos": [3, 8]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 19, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 13?", "why_question": "Why must Step 3 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 3 happen before Step 13?\n2. Why must Step 3 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 13 because the pumpkin needs to be cooked and mashed before brushing with honey.\"\n}", "relevant_step_nos": [3, 13]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 20, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 12?", "why_question": "Why must Step 3 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 3 happen before Step 12?\n2. Why must Step 3 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 12 because the pumpkin needs to be cooked and mashed before allowing the loaf to cool.\"\n}", "relevant_step_nos": [3, 12]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 21, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 10?", "why_question": "Why must Step 3 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 3 happen before Step 10?\n2. Why must Step 3 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 10 because the pumpkin needs to be cooked and mashed before it can be sprinkled with pumpkin seeds and baked.\"\n}", "relevant_step_nos": [3, 10]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 22, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 12?", "why_question": "Why must Step 4 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 4 happen before Step 12?\n2. Why must Step 4 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 12 because the teabread needs to cool in the tin for 10 minutes after baking, and the pumpkin mixture needs to be brushed with honey while still warm.\"\n}", "relevant_step_nos": [4, 12]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 23, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the pumpkin needs to be mashed and mixed with honey and sultanas before adding the flour and allspice.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 24, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 13?", "why_question": "Why must Step 4 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 4 happen before Step 13?\n2. Why must Step 4 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 13 because the teabread needs to be brushed with honey while still warm, and the pumpkin mixture needs to be prepared before baking the loaf.\"\n}", "relevant_step_nos": [4, 13]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 25, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 14?", "why_question": "Why must Step 4 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 4 happen before Step 14?\n2. Why must Step 4 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 involves mashing the pumpkin, which is a key ingredient in the teabread, so it must happen before serving the teabread.\"\n}", "relevant_step_nos": [4, 14]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 26, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 12?", "why_question": "Why must Step 8 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 8 happen before Step 12?\n2. Why must Step 8 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 12 because the loaf needs to be turned out onto a wire rack after cooling in the tin.\"\n}", "relevant_step_nos": [8, 12]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 27, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 10?", "why_question": "Why must Step 8 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 8 happen before Step 10?\n2. Why must Step 8 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 10 because the pumpkin mixture needs to be evenly mixed into the flour and butter mixture before baking.\" \n}", "relevant_step_nos": [8, 10]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 28, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 9 because the pumpkin mixture needs to be evenly mixed into the flour and butter mixture before it is transferred into the tin and smoothed.\" \n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 29, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 11?", "why_question": "Why must Step 8 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 8 happen before Step 11?\n2. Why must Step 8 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 11 because the pumpkin mixture needs to be evenly mixed into the flour and butter mixture before baking.\"\n}", "relevant_step_nos": [8, 11]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 30, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 12?", "why_question": "Why must Step 10 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 10 happen before Step 12?\n2. Why must Step 10 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 12 because the teabread needs to be turned out of the tin before it cools completely.\"\n}", "relevant_step_nos": [10, 12]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 31, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 12?", "why_question": "Why must Step 1 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 1 happen before Step 12?\n2. Why must Step 1 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 12 because the loaf needs to cool in the tin for 10 minutes before being turned out onto a wire rack.\"\n}", "relevant_step_nos": [1, 12]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 32, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 14?", "why_question": "Why must Step 1 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 1 happen before Step 14?\n2. Why must Step 1 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 14 because the oven needs to be preheated before baking the teabread.\"\n}", "relevant_step_nos": [1, 14]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 33, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 13?", "why_question": "Why must Step 1 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 1 happen before Step 13?\n2. Why must Step 1 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 13 because the loaf needs to cool in the tin for 10 minutes before being turned out onto a wire rack, and Step 1 involves preheating the oven for baking the loaf.\"\n}", "relevant_step_nos": [1, 13]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 34, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 11?", "why_question": "Why must Step 1 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 1 happen before Step 11?\n2. Why must Step 1 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 11 because the oven needs to be preheated before baking the teabread.\"\n}", "relevant_step_nos": [1, 11]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 35, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 9?", "why_question": "Why must Step 7 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 7 happen before Step 9?\n2. Why must Step 7 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 9 because the pumpkin mixture needs to be evenly mixed with the flour and butter before being placed in the tin and smoothed.\" \n}", "relevant_step_nos": [7, 9]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 36, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 11?", "why_question": "Why must Step 7 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 7 happen before Step 11?\n2. Why must Step 7 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 11 because the mixture needs to be prepared and put into the tin before baking.\"\n}", "relevant_step_nos": [7, 11]}
{"plan_idx": 46, "title": "spiced-pumpkin-teabread", "question_idx": 37, "steps": ["Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).", "Line a 900 g (2 lb) loaf tin with baking parchment.", "Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.", "Drain thoroughly, then mash with a potato masher until smooth.", "Add the honey and sultanas, and beat well.", "Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.", "Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.", "Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.", "Tip the mixture into the prepared tin and smooth the surface.", "Sprinkle with the pumpkin seeds.", "Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.", "Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.", "Gently brush with the 2 tsp honey while still warm.", "This teabread is best served slightly warm, or within a day of making."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 12?", "why_question": "Why must Step 7 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spiced-pumpkin-teabread\n\nProcedure:\n1. Preheat the oven to 180\u5c0aC (350\u5c0aF, gas mark 4).\n2. Line a 900 g (2 lb) loaf tin with baking parchment.\n3. Steam the pumpkin flesh, or cook it in just a little boiling water, for 10 minutes or until tender.\n4. Drain thoroughly, then mash with a potato masher until smooth.\n5. Add the honey and sultanas, and beat well.\n6. Sift the wholemeal and white flours and the allspice into a bowl, tipping in any bran left in the sieve.\n7. Rub in the butter with your fingertips until the mixture resembles fine breadcrumbs.\n8. Add the pumpkin mixture and beat well with a wooden spoon until evenly mixed.\n9. Tip the mixture into the prepared tin and smooth the surface.\n10. Sprinkle with the pumpkin seeds.\n11. Bake for 50 minutes or until the loaf is well risen, golden brown and firm to the touch.\n12. Allow the loaf to cool in the tin for 10 minutes, then turn it out onto a wire rack.\n13. Gently brush with the 2 tsp honey while still warm.\n14. This teabread is best served slightly warm, or within a day of making.\n\n\n1. Must Step 7 happen before Step 12?\n2. Why must Step 7 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 12 because the loaf needs to be baked before it can be allowed to cool in the tin.\"\n}", "relevant_step_nos": [7, 12]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 0, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 10?", "why_question": "Why must Step 8 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 8 happen before Step 10?\n2. Why must Step 8 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 10 because the pasta needs to be tossed with the anchovy mixture and chilli flakes before serving immediately.\"\n}", "relevant_step_nos": [8, 10]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 1, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the linguine needs to be cooked before it can be drained and tossed with the anchovy mixture.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 2, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 10?", "why_question": "Why must Step 5 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 5 happen before Step 10?\n2. Why must Step 5 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The pasta needs to be cooked before it can be served with the anchovy mixture.\"\n}", "relevant_step_nos": [5, 10]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 3, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 10?", "why_question": "Why must Step 7 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 7 happen before Step 10?\n2. Why must Step 7 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 10 because the linguine needs to be cooked and drained before being tossed with the anchovy mixture and served immediately.\"\n}", "relevant_step_nos": [7, 10]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 4, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 10?", "why_question": "Why must Step 2 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 2 happen before Step 10?\n2. Why must Step 2 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 10 because the vegetables need to be cooked before serving the dish immediately.\"\n}", "relevant_step_nos": [2, 10]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 5, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 3 because the anchovies and water are added to the cooked vegetables in Step 3.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 6, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 8?", "why_question": "Why must Step 2 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 2 happen before Step 8?\n2. Why must Step 2 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 8 because the linguine needs to be cooked while the anchovy mixture is being prepared.\"\n}", "relevant_step_nos": [2, 8]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 7, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 10?", "why_question": "Why must Step 9 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 9 happen before Step 10?\n2. Why must Step 9 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 is seasoning the pasta with black pepper before serving in Step 10.\"\n}", "relevant_step_nos": [9, 10]}
{"plan_idx": 47, "title": "spicy-tomato-anchovy-pasta", "question_idx": 8, "steps": ["Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;", "cook until lightly browned.", "Add anchovies and water, cover and simmer for 4 to 5 minutes.", "Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.", "While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.", "Add linguine and cook until al dente, about 7 to 8 minutes;", "drain.", "Toss with anchovy mixture and chilli flakes.", "If desired, season with black pepper.", "Serve immediately."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: spicy-tomato-anchovy-pasta\n\nProcedure:\n1. Heat 6 tablespoons olive oil in a large frying pan over medium heat, then stir in garlic, broccoli and mushrooms;\n2. cook until lightly browned.\n3. Add anchovies and water, cover and simmer for 4 to 5 minutes.\n4. Stir in spring onions, tomatoes and parsley and cover again, simmering until vegetables are soft, about 3 to 4 minutes.\n5. While the vegetables are cooking, bring a large pot of water and one teaspoon of olive oil to the boil.\n6. Add linguine and cook until al dente, about 7 to 8 minutes;\n7. drain.\n8. Toss with anchovy mixture and chilli flakes.\n9. If desired, season with black pepper.\n10. Serve immediately.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because the anchovies and water need time to simmer and infuse the flavors before adding the spring onions, tomatoes, and parsley.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 0, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 6?", "why_question": "Why must Step 4 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 4 happen before Step 6?\n2. Why must Step 4 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 6 to allow the strawberries to absorb the Pimm's flavor before garnishing with mint leaves.\"\n}", "relevant_step_nos": [4, 6]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 1, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because the strawberries need to be coated with sugar before garnishing with mint leaves.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 2, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the strawberries to absorb the sugar before adding the Pimm's.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 3, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 6?", "why_question": "Why must Step 1 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 1 happen before Step 6?\n2. Why must Step 1 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to prepare the strawberries before garnishing with mint leaves in Step 6.\"\n}", "relevant_step_nos": [1, 6]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 4, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 5?", "why_question": "Why must Step 1 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 1 happen before Step 5?\n2. Why must Step 1 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 5 because the strawberries need to be prepared and coated with sugar before adding Pimm's and letting it stand for 30 minutes.\"\n}", "relevant_step_nos": [1, 5]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 5, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 allows the strawberries to absorb the sugar before adding the Pimm's in Step 4.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 6, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 to allow the strawberries to absorb the sugar and flavor from the Pimm's.\" \n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 7, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 6 to allow the strawberries to absorb the sugar and develop flavor before garnishing with mint leaves.\"\n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 48, "title": "strawberries-in-pimm-s", "question_idx": 8, "steps": ["Place the strawberries in a large bowl.", "Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.", "Let stand at room temperature for 15 minutes, stirring occasionally.", "Add the Pimm's, and stir once more.", "Let stand for another 30 minutes at room temperature.", "Garnish with mint leaves, if desired."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberries-in-pimm-s\n\nProcedure:\n1. Place the strawberries in a large bowl.\n2. Sprinkle the sugar evenly over the strawberries, and stir to evenly coat all of the fruit.\n3. Let stand at room temperature for 15 minutes, stirring occasionally.\n4. Add the Pimm's, and stir once more.\n5. Let stand for another 30 minutes at room temperature.\n6. Garnish with mint leaves, if desired.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 to allow the strawberries to absorb the Pimm's flavor before garnishing with mint leaves.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 0, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the strawberries need to be spread over the sponge fingers before adding the grated chocolate and custard mixture.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 1, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 10?", "why_question": "Why must Step 6 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 6 happen before Step 10?\n2. Why must Step 6 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 10 because it is part of the layering process and needs to be completed before the trifle is covered and refrigerated.\"\n}", "relevant_step_nos": [6, 10]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 2, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 8 to ensure the layers are properly assembled in the trifle.\"\n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 3, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 6?", "why_question": "Why must Step 4 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 4 happen before Step 6?\n2. Why must Step 4 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 6 because the soaked sponge fingers need to be lined in the glass bowl before adding the strawberries.\"\n}", "relevant_step_nos": [4, 6]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 4, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 7?", "why_question": "Why must Step 4 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 4 happen before Step 7?\n2. Why must Step 4 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 7 because the soaked sponge fingers need to be layered with the strawberries and grated chocolate before adding the custard mixture.\"\n}", "relevant_step_nos": [4, 7]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 5, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the sponge fingers need to be brushed with coffee before being lined in the glass bowl.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 6, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 7 because the custard mixture needs to be prepared before it can be used in the layers.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 7, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 10?", "why_question": "Why must Step 1 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 1 happen before Step 10?\n2. Why must Step 1 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to prepare the strawberries for garnish before assembling the trifle.\"\n}", "relevant_step_nos": [1, 10]}
{"plan_idx": 49, "title": "strawberry-tiramisu-trifle", "question_idx": 8, "steps": ["Set 3 strawberries for the garnish aside, and cut the rest into slices.", "In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.", "Then stir in the custard and finally fold in the whipped cream.", "Brush the sponge fingers with the remaining coffee.", "Line a glass bowl with one layer of the soaked sponge fingers.", "Spread half of the strawberries over the sponge fingers.", "Sprinkle half of the grated chocolate over that and then half of the custard mixture.", "Repeat the layers in order again.", "Cut the remaining 3 strawberries in half and lay them on top of the trifle.", "Cover and leave in the fridge for at least 4 hours, best overnight."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 9?", "why_question": "Why must Step 7 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: strawberry-tiramisu-trifle\n\nProcedure:\n1. Set 3 strawberries for the garnish aside, and cut the rest into slices.\n2. In a large bowl, beat the cream cheese with a hand whisk until creamy, gradually adding 2 tablespoons of the coffee.\n3. Then stir in the custard and finally fold in the whipped cream.\n4. Brush the sponge fingers with the remaining coffee.\n5. Line a glass bowl with one layer of the soaked sponge fingers.\n6. Spread half of the strawberries over the sponge fingers.\n7. Sprinkle half of the grated chocolate over that and then half of the custard mixture.\n8. Repeat the layers in order again.\n9. Cut the remaining 3 strawberries in half and lay them on top of the trifle.\n10. Cover and leave in the fridge for at least 4 hours, best overnight.\n\n\n1. Must Step 7 happen before Step 9?\n2. Why must Step 7 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 9 because the strawberries are part of the layering process and need to be added before the final garnish.\" \n}", "relevant_step_nos": [7, 9]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 0, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The tomatoes need to be prepared before they can be used in the mixture for the gammon steaks.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 1, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 7?", "why_question": "Why must Step 2 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 2 happen before Step 7?\n2. Why must Step 2 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 2 does not need to happen before Step 7 as they are independent steps in the procedure.\"\n}", "relevant_step_nos": [2, 7]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 2, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the tomatoes to cool and be easier to handle for peeling and chopping.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 3, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 8?", "why_question": "Why must Step 3 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 3 happen before Step 8?\n2. Why must Step 3 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 8 because the onions need to be prepared and placed in the casserole dish before pouring the cider over the steaks.\"\n}", "relevant_step_nos": [3, 8]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 4, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The onions need to be chopped before being added to the casserole dish in Step 5.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 5, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 7 because the onions need to be prepared and placed in the casserole dish before the gammon steaks are wrapped and placed in the dish.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 6, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 8 because the gammon steaks need to be prepared and placed in the dish before pouring the cider over them and covering with a lid or tin foil.\"\n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 50, "title": "tangy-gammon-in-cider", "question_idx": 7, "steps": ["Preheat the oven to 200 degrees /gas mark 6.", "Start by piercing each tomato once and covering with boiling water to loosen the skins.", "Drain the onions and chop each one in half.", "Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.", "Put the onions, tomatoes and apples into a casserole dish.", "Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.", "Place in the dish and repeat with the other gammon steaks.", "Pour cider over the steaks, cover the top with a lid or tin foil.", "Cook for 1 hour."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tangy-gammon-in-cider\n\nProcedure:\n1. Preheat the oven to 200 degrees /gas mark 6.\n2. Start by piercing each tomato once and covering with boiling water to loosen the skins.\n3. Drain the onions and chop each one in half.\n4. Once the tomatoes have been in the water for 5 minutes, drain, then remove the skins and coarsely chop into largish pieces.\n5. Put the onions, tomatoes and apples into a casserole dish.\n6. Take a gammon steak, lay flat, and spoon some of the mixture into the middle, then wrap the sides of the steak over the top and secure with a small skewer or piece of raw spaghetti.\n7. Place in the dish and repeat with the other gammon steaks.\n8. Pour cider over the steaks, cover the top with a lid or tin foil.\n9. Cook for 1 hour.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The tomatoes need to be chopped and added to the casserole dish before cooking for 1 hour.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 0, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 5?", "why_question": "Why must Step 1 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 1 happen before Step 5?\n2. Why must Step 1 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 5 to prepare the serving plate before placing the salad on it.\"\n}", "relevant_step_nos": [1, 5]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 1, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 4?", "why_question": "Why must Step 1 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 1 happen before Step 4?\n2. Why must Step 1 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 4 because the serving plate needs to be prepared before drizzling the vinegar and oil over the salad.\"\n}", "relevant_step_nos": [1, 4]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 2, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 3 can happen before Step 1 as slicing the tomatoes and scattering the basil can be done in any order.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 3, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to have a plate ready for the sliced tomatoes.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 4, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 4?", "why_question": "Why must Step 3 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 3 happen before Step 4?\n2. Why must Step 3 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 4 because the salt needs to be sprinkled on the tomatoes before drizzling with vinegar and oil.\"\n}", "relevant_step_nos": [3, 4]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 5, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 to allow the flavors to meld together while the salad sits at room temperature.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 6, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 3?", "why_question": "Why must Step 2 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 2 happen before Step 3?\n2. Why must Step 2 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 3 because the basil leaves need to be scattered over the sliced tomatoes.\"\n}", "relevant_step_nos": [2, 3]}
{"plan_idx": 51, "title": "tomato and basil salad", "question_idx": 7, "steps": ["Grab an attractive serving plate; flat glass or black works nicely.", "Slice the tomatoes thinly and scatter onto plate.", "Sprinkle with salt, then scatter all over the basil leaves.", "Drizzle over the vinegar and oil.", "Cover with cling film and leave at room temperature until ready to serve."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 5?", "why_question": "Why must Step 2 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tomato and basil salad\n\nProcedure:\n1. Grab an attractive serving plate; flat glass or black works nicely.\n2. Slice the tomatoes thinly and scatter onto plate.\n3. Sprinkle with salt, then scatter all over the basil leaves.\n4. Drizzle over the vinegar and oil.\n5. Cover with cling film and leave at room temperature until ready to serve.\n\n\n1. Must Step 2 happen before Step 5?\n2. Why must Step 2 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 5 because the tomatoes need to be sliced and arranged on the plate before covering with cling film and leaving at room temperature.\"\n}", "relevant_step_nos": [2, 5]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 0, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 7 because the tortellini needs to be fried before serving.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 1, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 5?", "why_question": "Why must Step 3 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 3 happen before Step 5?\n2. Why must Step 3 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 5 because the tortellini needs to be fried before adding the almond butter and other ingredients.\"\n}", "relevant_step_nos": [3, 5]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 2, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 3?", "why_question": "Why must Step 1 happen before Step 3?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 1 happen before Step 3?\n2. Why must Step 1 happen before Step 3?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 needs to happen before Step 3 because the tortellini needs to be boiled before it can be fried.\"\n}", "relevant_step_nos": [1, 3]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 3, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 5?", "why_question": "Why must Step 1 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 1 happen before Step 5?\n2. Why must Step 1 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 needs to happen before Step 5 because the tortellini needs to be cooked before it can be fried and mixed with almond butter.\"\n}", "relevant_step_nos": [1, 5]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 4, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 6?", "why_question": "Why must Step 1 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 1 happen before Step 6?\n2. Why must Step 1 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 needs to happen before Step 6 because the tortellini needs to be cooked before it can be fried and combined with the almond butter.\"\n}", "relevant_step_nos": [1, 6]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 5, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 7?", "why_question": "Why must Step 2 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 2 happen before Step 7?\n2. Why must Step 2 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 7 because the tortellini needs to be cooked before serving.\"\n}", "relevant_step_nos": [2, 7]}
{"plan_idx": 52, "title": "tortellini-fritto-with-almond-butter", "question_idx": 6, "steps": ["Bring a saucepan of water to the boil.", "Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.", "In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.", "If you slightly undercooked your tortellini then it's less likely to stick to the pan.", "Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.", "Continue to gently fry and stir the tortellini until golden brown.", "Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.", "Enjoy!"], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 7?", "why_question": "Why must Step 6 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: tortellini-fritto-with-almond-butter\n\nProcedure:\n1. Bring a saucepan of water to the boil.\n2. Add the tortellini and cook according to the instructions on the packet, however I recommend you slightly under cook it and drain.\n3. In a non-stick shallow pan, heat some olive oil and gently fry the tortellini.\n4. If you slightly undercooked your tortellini then it's less likely to stick to the pan.\n5. Add some honey, salt to taste and around 2 tablespoons of the almond butter to the pan.\n6. Continue to gently fry and stir the tortellini until golden brown.\n7. Serve the tortellini with a drizzle of olive oil, freshly grated Parmesan cheese and green leaf salad of your choice.\n8. Enjoy!\n\n\n1. Must Step 6 happen before Step 7?\n2. Why must Step 6 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 7 because the tortellini needs to be fried until golden brown before serving.\"\n}", "relevant_step_nos": [6, 7]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 0, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 11?", "why_question": "Why must Step 5 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 5 happen before Step 11?\n2. Why must Step 5 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 11 because the cream mixture needs to be prepared before arranging the vegetables in the terrine.\"\n}", "relevant_step_nos": [5, 11]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 1, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the cream and yogurt need to be brought to a boil before adding the chopped sorrel and gelatin.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 2, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 12?", "why_question": "Why must Step 5 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 5 happen before Step 12?\n2. Why must Step 5 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 12 because the cream mixture needs to be poured over the arranged vegetables before refrigerating the terrine.\"\n}", "relevant_step_nos": [5, 12]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 3, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 13?", "why_question": "Why must Step 5 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 5 happen before Step 13?\n2. Why must Step 5 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 13 because the cream mixture needs to be brought to a boil and then cooled before pouring it over the vegetables and refrigerating the terrine.\"\n}", "relevant_step_nos": [5, 13]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 4, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 8?", "why_question": "Why must Step 5 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 5 happen before Step 8?\n2. Why must Step 5 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 8 because the cream mixture needs to be brought to a boil before adding the chopped sorrel and gelatin.\"\n}", "relevant_step_nos": [5, 8]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 5, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 14?", "why_question": "Why must Step 5 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 5 happen before Step 14?\n2. Why must Step 5 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 needs to happen before Step 14 because the cream mixture needs to be prepared and poured into the mould before refrigerating and releasing the terrine.\" \n}", "relevant_step_nos": [5, 14]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 6, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 15?", "why_question": "Why must Step 5 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 5 happen before Step 15?\n2. Why must Step 5 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 15 because the cream mixture needs to be brought to a boil and then cooled before pouring it into the mould and refrigerating.\" \n}", "relevant_step_nos": [5, 15]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 7, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 15?", "why_question": "Why must Step 10 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 10 happen before Step 15?\n2. Why must Step 10 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 15 because the terrine needs to be refrigerated to set before it can be served chilled.\"\n}", "relevant_step_nos": [10, 15]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 8, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 11?", "why_question": "Why must Step 10 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 10 happen before Step 11?\n2. Why must Step 10 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 11 because pouring the cream mixture into the mould and refrigerating it will create the base for arranging the vegetables in the terrine.\"\n}", "relevant_step_nos": [10, 11]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 9, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 15?", "why_question": "Why must Step 6 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 6 happen before Step 15?\n2. Why must Step 6 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 15 because the cream mixture needs to set and cool before the terrine can be served chilled.\"\n}", "relevant_step_nos": [6, 15]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 10, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 13?", "why_question": "Why must Step 6 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 6 happen before Step 13?\n2. Why must Step 6 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 is necessary to ensure that the gelatin is mixed well with the cream mixture before refrigerating the terrine in Step 13.\"\n}", "relevant_step_nos": [6, 13]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 11, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 14?", "why_question": "Why must Step 6 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 6 happen before Step 14?\n2. Why must Step 6 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 14 because the gelatin needs time to set and firm up the terrine before it can be released from the mould.\"\n}", "relevant_step_nos": [6, 14]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 12, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because the chopped sorrel needs to be added to the cream mixture before pouring it into the mould.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 13, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 11?", "why_question": "Why must Step 4 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 4 happen before Step 11?\n2. Why must Step 4 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 11 because the chopped sorrel from Step 4 needs to be arranged with the vegetables in Step 11.\"\n}", "relevant_step_nos": [4, 11]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 14, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 14?", "why_question": "Why must Step 4 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 4 happen before Step 14?\n2. Why must Step 4 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 14 because the sorrel needs to be cooked and set aside before the terrine is refrigerated and served.\"\n}", "relevant_step_nos": [4, 14]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 15, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 13?", "why_question": "Why must Step 4 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 4 happen before Step 13?\n2. Why must Step 4 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"no\",\n  \"why_answer\": \"Step 4 does not need to happen before Step 13 as it is not directly related to the setting of the terrine.\"\n}", "relevant_step_nos": [4, 13]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 16, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 14?", "why_question": "Why must Step 11 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 11 happen before Step 14?\n2. Why must Step 11 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 14 because the arrangement of the vegetables needs to be set before the terrine is refrigerated and then released from the mould.\"\n}", "relevant_step_nos": [11, 14]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 17, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 15?", "why_question": "Why must Step 11 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 11 happen before Step 15?\n2. Why must Step 11 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 15 because the vegetables need to be arranged in the mould before refrigerating and serving the terrine.\"\n}", "relevant_step_nos": [11, 15]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 18, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 11 happen before Step 12?", "why_question": "Why must Step 11 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 11 happen before Step 12?\n2. Why must Step 11 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 11 must happen before Step 12 because the vegetables need to be arranged before pouring the remaining cream mixture on top.\"\n}", "relevant_step_nos": [11, 12]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 19, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 15?", "why_question": "Why must Step 9 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 9 happen before Step 15?\n2. Why must Step 9 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 15 because the terrine needs to be refrigerated for several hours until firm before it can be served chilled.\"\n}", "relevant_step_nos": [9, 15]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 20, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 14?", "why_question": "Why must Step 9 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 9 happen before Step 14?\n2. Why must Step 9 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 14 because the terrine needs to be refrigerated and set before it can be removed from the mould.\"\n}", "relevant_step_nos": [9, 14]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 21, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 13?", "why_question": "Why must Step 9 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 9 happen before Step 13?\n2. Why must Step 9 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 13 because the cream mixture needs to set and firm up before the terrine can be refrigerated for several hours.\"\n}", "relevant_step_nos": [9, 13]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 22, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 9?", "why_question": "Why must Step 1 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 1 happen before Step 9?\n2. Why must Step 1 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 softens the gelatin, making it easier to mix into the cream mixture in Step 9.\"\n}", "relevant_step_nos": [1, 9]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 23, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 15?", "why_question": "Why must Step 1 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 1 happen before Step 15?\n2. Why must Step 1 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to soften the gelatin before it can be used in the terrine, which is essential for the texture of the dish.\"\n}", "relevant_step_nos": [1, 15]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 24, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 8?", "why_question": "Why must Step 1 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 1 happen before Step 8?\n2. Why must Step 1 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 needs to happen before Step 8 because the gelatin needs to be softened before it can be added to the cream mixture in Step 8.\"\n}", "relevant_step_nos": [1, 8]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 25, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 15?", "why_question": "Why must Step 8 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 8 happen before Step 15?\n2. Why must Step 8 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 15 because the terrine needs to cool and set in the refrigerator for several hours before it can be served chilled.\"\n}", "relevant_step_nos": [8, 15]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 26, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 14?", "why_question": "Why must Step 12 happen before Step 14?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 12 happen before Step 14?\n2. Why must Step 12 happen before Step 14?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 12 must happen before Step 14 because pouring the remaining cream mixture on top needs to happen before refrigerating the terrine for several hours until firm, and before attempting to release the terrine from the mould.\" \n}", "relevant_step_nos": [12, 14]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 27, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 11?", "why_question": "Why must Step 3 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 3 happen before Step 11?\n2. Why must Step 3 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To ensure that the vegetables are cooled and ready to be arranged in the terrine before pouring the cream mixture on top.\"\n}", "relevant_step_nos": [3, 11]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 28, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 14 happen before Step 15?", "why_question": "Why must Step 14 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 14 happen before Step 15?\n2. Why must Step 14 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 14 must happen before Step 15 because immersing the mould in boiling water and turning it over is necessary to release the terrine from the mould before serving.\"\n}", "relevant_step_nos": [14, 15]}
{"plan_idx": 53, "title": "vegetable-terrine", "question_idx": 29, "steps": ["Add the gelatin in a bowl of cold water until soft.", "Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).", "Run the vegetables under cold water to stop them cooking.", "Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.", "In a saucepan, bring cream and yogurt to the boil.", "Add the chopped sorrel and gelatin.", "Mix well and remove from heat.", "Set aside and cool for a few minutes.", "Pour approximately 1cm of the cream mixture into a mould and refrigerate.", "It is better to use a silicone mould, it will be easier to remove for serving.", "Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.", "Pour the remaining cream mixture on top.", "Refrigerate the terrine for several hours until firm.", "If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.", "Serve chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 15?", "why_question": "Why must Step 2 happen before Step 15?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: vegetable-terrine\n\nProcedure:\n1. Add the gelatin in a bowl of cold water until soft.\n2. Heat the stock and cook the vegetables one after the other in the stock (15 minutes for the carrots and beans, 10 minutes for courgettes and celeriac, 5 minutes for the peas).\n3. Run the vegetables under cold water to stop them cooking.\n4. Wash the sorrel, add the olive oil to a frying pan and cook for 3 minutes, chop then set aside.\n5. In a saucepan, bring cream and yogurt to the boil.\n6. Add the chopped sorrel and gelatin.\n7. Mix well and remove from heat.\n8. Set aside and cool for a few minutes.\n9. Pour approximately 1cm of the cream mixture into a mould and refrigerate.\n10. It is better to use a silicone mould, it will be easier to remove for serving.\n11. Arrange the vegetables, alternating according to shape, starting with the matchstick vegetables on the bottom and peas in the middle.\n12. Pour the remaining cream mixture on top.\n13. Refrigerate the terrine for several hours until firm.\n14. If you did'nt use a silicone mould, immerse the mould in boiling water then turn it over - it should release.\n15. Serve chilled.\n\n\n1. Must Step 2 happen before Step 15?\n2. Why must Step 2 happen before Step 15?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 is necessary to cook the vegetables before assembling the terrine, which is essential for the final dish.\"\n}", "relevant_step_nos": [2, 15]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 0, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 13?", "why_question": "Why must Step 10 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 10 happen before Step 13?\n2. Why must Step 10 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 13 because the cheesecake needs to be baked before it can be refrigerated.\"\n}", "relevant_step_nos": [10, 13]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 1, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 11?", "why_question": "Why must Step 10 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 10 happen before Step 11?\n2. Why must Step 10 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 must happen before Step 11 to ensure that the baking dish is filled with boiling water before placing the cheesecake in the oven.\"\n}", "relevant_step_nos": [10, 11]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 2, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 11?", "why_question": "Why must Step 2 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 2 happen before Step 11?\n2. Why must Step 2 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 11 because the biscuit crumbs need to be prepared and the base of the cheesecake needs to be formed before baking the cheesecake.\"\n}", "relevant_step_nos": [2, 11]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 3, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 9?", "why_question": "Why must Step 2 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 2 happen before Step 9?\n2. Why must Step 2 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 9 because the biscuit base needs to be prepared before pouring the cheesecake mixture into the tin.\"\n}", "relevant_step_nos": [2, 9]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 4, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 12?", "why_question": "Why must Step 2 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 2 happen before Step 12?\n2. Why must Step 2 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 12 because the crumbs need to be prepared and the base of the cheesecake needs to be set before the cheesecake is baked and cooled.\"\n}", "relevant_step_nos": [2, 12]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 5, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 13?", "why_question": "Why must Step 6 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 6 happen before Step 13?\n2. Why must Step 6 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 13 because the cheesecake needs to be mixed and beaten before it is refrigerated to set.\"\n}", "relevant_step_nos": [6, 13]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 6, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 9?", "why_question": "Why must Step 6 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 6 happen before Step 9?\n2. Why must Step 6 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 must happen before Step 9 because the cheesecake mixture needs to be prepared before pouring it into the biscuit base.\"\n}", "relevant_step_nos": [6, 9]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 7, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 12?", "why_question": "Why must Step 3 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 3 happen before Step 12?\n2. Why must Step 3 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 12 because the crumbs need to be mixed with the vanilla seeds before the cheesecake is cooled and refrigerated.\"\n}", "relevant_step_nos": [3, 12]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 8, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 12 happen before Step 13?", "why_question": "Why must Step 12 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 12 happen before Step 13?\n2. Why must Step 12 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the cheesecake to cool down slowly and set properly before refrigerating.\"\n}", "relevant_step_nos": [12, 13]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 9, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 13?", "why_question": "Why must Step 9 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 9 happen before Step 13?\n2. Why must Step 9 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 13 because the cheesecake needs to be poured into the biscuit base before it can be covered and refrigerated.\"\n}", "relevant_step_nos": [9, 13]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 10, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 9 because the mixture needs to be prepared and poured into the biscuit base before baking.\"\n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 11, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 12?", "why_question": "Why must Step 8 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 8 happen before Step 12?\n2. Why must Step 8 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 must happen before Step 12 because the cheesecake needs to be blended and smooth before cooling down in the oven.\"\n}", "relevant_step_nos": [8, 12]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 12, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 9?", "why_question": "Why must Step 7 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 7 happen before Step 9?\n2. Why must Step 7 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 must happen before Step 9 because the mixture needs to be poured into the biscuit base before baking.\"\n}", "relevant_step_nos": [7, 9]}
{"plan_idx": 54, "title": "very-vanilla-ry-cheesecake", "question_idx": 13, "steps": ["Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.", "Place the biscuits into a plastic bag and crush with a rolling pin.", "Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.", "Drizzle the crumbs with melted butter and stir to combine.", "Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.", "Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.", "Beat in the eggs one at a time, incorporating each egg before adding the next one.", "Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.", "Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.", "Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.", "Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.", "Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.", "Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 13?", "why_question": "Why must Step 1 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: very-vanilla-ry-cheesecake\n\nProcedure:\n1. Preheat oven to 150 C / Gas 2 and lightly butter a 26cm springform tin.\n2. Place the biscuits into a plastic bag and crush with a rolling pin.\n3. Pour the crumbs into a bowl and mix with the scraped seeds of 1 vanilla bean.\n4. Drizzle the crumbs with melted butter and stir to combine.\n5. Press the crumb mixture firmly into the bottom and up the sides of the springform tin and set aside.\n6. Place the softened cream cheese and sugar into the work bowl of an electric mixer and beat for about 3 minutes on Medium speed, until the mixture is smooth.\n7. Beat in the eggs one at a time, incorporating each egg before adding the next one.\n8. Add the scraped seeds of 2 vanilla beans, vanilla extract, soured cream and double cream and beat on Medium-High speed for about 1 minute, until the mixture is blended and smooth.\n9. Pour the mixture into the biscuit base, leaving about 5cm of space from the top of the tin for the cake to expand as it bakes.\n10. Place a baking dish onto the bottom rack of the preheated oven and fill it halfway with boiling water.\n11. Place the cheesecake onto the middle rack, above the tin of boiling water and bake until a small knife inserted into the middle of the cheesecake comes out clean, about 1 hour and 20 minutes.\n12. Turn the oven off and let the cheesecake cool down inside the oven for 1 hour.\n13. Cover the cheesecake with cling film, then foil and refrigerate at least 4 hours or overnight before serving.\n\n\n1. Must Step 1 happen before Step 13?\n2. Why must Step 1 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 must happen before Step 13 because the oven needs to be preheated before the cheesecake can be baked, and the cooling process begins after the baking is complete.\"\n}", "relevant_step_nos": [1, 13]}
{"plan_idx": 55, "title": "warm-puy-lentil-and-tomato-salad", "question_idx": 0, "steps": ["Bring a large pan of water to the boil;", "add lentils and cook over low heat for 30 minutes.", "Put tomatoes in a small bowl and season with salt.", "Set aside for 15 minutes.", "Drain lentils and tomatoes and mix together.", "Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 6?", "why_question": "Why must Step 3 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: warm-puy-lentil-and-tomato-salad\n\nProcedure:\n1. Bring a large pan of water to the boil;\n2. add lentils and cook over low heat for 30 minutes.\n3. Put tomatoes in a small bowl and season with salt.\n4. Set aside for 15 minutes.\n5. Drain lentils and tomatoes and mix together.\n6. Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled.\n\n\n1. Must Step 3 happen before Step 6?\n2. Why must Step 3 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 6 because the tomatoes need to be seasoned and set aside before being mixed with the lentils.\"\n}", "relevant_step_nos": [3, 6]}
{"plan_idx": 55, "title": "warm-puy-lentil-and-tomato-salad", "question_idx": 1, "steps": ["Bring a large pan of water to the boil;", "add lentils and cook over low heat for 30 minutes.", "Put tomatoes in a small bowl and season with salt.", "Set aside for 15 minutes.", "Drain lentils and tomatoes and mix together.", "Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 6?", "why_question": "Why must Step 2 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: warm-puy-lentil-and-tomato-salad\n\nProcedure:\n1. Bring a large pan of water to the boil;\n2. add lentils and cook over low heat for 30 minutes.\n3. Put tomatoes in a small bowl and season with salt.\n4. Set aside for 15 minutes.\n5. Drain lentils and tomatoes and mix together.\n6. Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled.\n\n\n1. Must Step 2 happen before Step 6?\n2. Why must Step 2 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 6 because the lentils need to be cooked before they can be mixed with the tomatoes and served.\"\n}", "relevant_step_nos": [2, 6]}
{"plan_idx": 55, "title": "warm-puy-lentil-and-tomato-salad", "question_idx": 2, "steps": ["Bring a large pan of water to the boil;", "add lentils and cook over low heat for 30 minutes.", "Put tomatoes in a small bowl and season with salt.", "Set aside for 15 minutes.", "Drain lentils and tomatoes and mix together.", "Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 2?", "why_question": "Why must Step 1 happen before Step 2?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: warm-puy-lentil-and-tomato-salad\n\nProcedure:\n1. Bring a large pan of water to the boil;\n2. add lentils and cook over low heat for 30 minutes.\n3. Put tomatoes in a small bowl and season with salt.\n4. Set aside for 15 minutes.\n5. Drain lentils and tomatoes and mix together.\n6. Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled.\n\n\n1. Must Step 1 happen before Step 2?\n2. Why must Step 1 happen before Step 2?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to bring the water to a boil before adding the lentils.\"\n}", "relevant_step_nos": [1, 2]}
{"plan_idx": 55, "title": "warm-puy-lentil-and-tomato-salad", "question_idx": 3, "steps": ["Bring a large pan of water to the boil;", "add lentils and cook over low heat for 30 minutes.", "Put tomatoes in a small bowl and season with salt.", "Set aside for 15 minutes.", "Drain lentils and tomatoes and mix together.", "Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 5?", "why_question": "Why must Step 4 happen before Step 5?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: warm-puy-lentil-and-tomato-salad\n\nProcedure:\n1. Bring a large pan of water to the boil;\n2. add lentils and cook over low heat for 30 minutes.\n3. Put tomatoes in a small bowl and season with salt.\n4. Set aside for 15 minutes.\n5. Drain lentils and tomatoes and mix together.\n6. Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled.\n\n\n1. Must Step 4 happen before Step 5?\n2. Why must Step 4 happen before Step 5?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 5 because the tomatoes need to be drained before being mixed with the lentils.\"\n}", "relevant_step_nos": [4, 5]}
{"plan_idx": 55, "title": "warm-puy-lentil-and-tomato-salad", "question_idx": 4, "steps": ["Bring a large pan of water to the boil;", "add lentils and cook over low heat for 30 minutes.", "Put tomatoes in a small bowl and season with salt.", "Set aside for 15 minutes.", "Drain lentils and tomatoes and mix together.", "Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 6?", "why_question": "Why must Step 5 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: warm-puy-lentil-and-tomato-salad\n\nProcedure:\n1. Bring a large pan of water to the boil;\n2. add lentils and cook over low heat for 30 minutes.\n3. Put tomatoes in a small bowl and season with salt.\n4. Set aside for 15 minutes.\n5. Drain lentils and tomatoes and mix together.\n6. Distribute mixture evenly in 6 individual bowls or ramekins, sprinkle with chopped parsley and serve warm or chilled.\n\n\n1. Must Step 5 happen before Step 6?\n2. Why must Step 5 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 6 because the lentils and tomatoes need to be mixed together before being distributed into individual bowls or ramekins.\"\n}", "relevant_step_nos": [5, 6]}
{"plan_idx": 56, "title": "white bread", "question_idx": 0, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 7?", "why_question": "Why must Step 5 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 5 happen before Step 7?\n2. Why must Step 5 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 7 to allow the dough to rise properly.\"\n}", "relevant_step_nos": [5, 7]}
{"plan_idx": 56, "title": "white bread", "question_idx": 1, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 12?", "why_question": "Why must Step 5 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 5 happen before Step 12?\n2. Why must Step 5 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 5 must happen before Step 12 because kneading the dough is necessary before allowing it to rise for the second time.\"\n}", "relevant_step_nos": [5, 12]}
{"plan_idx": 56, "title": "white bread", "question_idx": 2, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 5 happen before Step 11?", "why_question": "Why must Step 5 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 5 happen before Step 11?\n2. Why must Step 5 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"The dough needs to be kneaded and smooth before shaping into loaves.\"\n}", "relevant_step_nos": [5, 11]}
{"plan_idx": 56, "title": "white bread", "question_idx": 3, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 6 happen before Step 8?", "why_question": "Why must Step 6 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 6 happen before Step 8?\n2. Why must Step 6 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 6 is necessary to coat the dough with oil before allowing it to rise in Step 8.\"\n}", "relevant_step_nos": [6, 8]}
{"plan_idx": 56, "title": "white bread", "question_idx": 4, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 9 happen before Step 11?", "why_question": "Why must Step 9 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 9 happen before Step 11?\n2. Why must Step 9 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 9 must happen before Step 11 to ensure that the dough is properly prepared and ready to be shaped into loaves.\"\n}", "relevant_step_nos": [9, 11]}
{"plan_idx": 56, "title": "white bread", "question_idx": 5, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 10?", "why_question": "Why must Step 3 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 3 happen before Step 10?\n2. Why must Step 3 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 10 because mixing salt and oil into the yeast mixture is necessary before kneading the dough and dividing it in half.\"\n}", "relevant_step_nos": [3, 10]}
{"plan_idx": 56, "title": "white bread", "question_idx": 6, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 7?", "why_question": "Why must Step 3 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 3 happen before Step 7?\n2. Why must Step 3 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 7 because the yeast needs time to proof and create a creamy foam before allowing the dough to rise.\"\n}", "relevant_step_nos": [3, 7]}
{"plan_idx": 56, "title": "white bread", "question_idx": 7, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 8?", "why_question": "Why must Step 3 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 3 happen before Step 8?\n2. Why must Step 3 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 8 because mixing salt and oil into the yeast mixture is necessary before allowing the dough to rise.\"\n}", "relevant_step_nos": [3, 8]}
{"plan_idx": 56, "title": "white bread", "question_idx": 8, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 11?", "why_question": "Why must Step 3 happen before Step 11?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 3 happen before Step 11?\n2. Why must Step 3 happen before Step 11?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 11 because mixing salt and oil into the yeast mixture is necessary before shaping the dough into loaves.\"\n}", "relevant_step_nos": [3, 11]}
{"plan_idx": 56, "title": "white bread", "question_idx": 9, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 3 happen before Step 13?", "why_question": "Why must Step 3 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 3 happen before Step 13?\n2. Why must Step 3 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 3 must happen before Step 13 because mixing salt and oil into the yeast mixture is necessary for the bread to rise properly and have the right texture.\"\n}", "relevant_step_nos": [3, 13]}
{"plan_idx": 56, "title": "white bread", "question_idx": 10, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 9?", "why_question": "Why must Step 7 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 7 happen before Step 9?\n2. Why must Step 7 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 allows the dough to rise, making it easier to punch down in Step 9.\"\n}", "relevant_step_nos": [7, 9]}
{"plan_idx": 56, "title": "white bread", "question_idx": 11, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 12?", "why_question": "Why must Step 7 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 7 happen before Step 12?\n2. Why must Step 7 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 allows the dough to rise before shaping it into loaves in Step 12.\"\n}", "relevant_step_nos": [7, 12]}
{"plan_idx": 56, "title": "white bread", "question_idx": 12, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 8?", "why_question": "Why must Step 7 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 7 happen before Step 8?\n2. Why must Step 7 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"To allow the dough to rise until doubled in bulk.\"\n}", "relevant_step_nos": [7, 8]}
{"plan_idx": 56, "title": "white bread", "question_idx": 13, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 7 happen before Step 10?", "why_question": "Why must Step 7 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 7 happen before Step 10?\n2. Why must Step 7 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 7 allows the dough to rise and become lighter, making it easier to knead in Step 10.\"\n}", "relevant_step_nos": [7, 10]}
{"plan_idx": 56, "title": "white bread", "question_idx": 14, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 4?", "why_question": "Why must Step 2 happen before Step 4?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 2 happen before Step 4?\n2. Why must Step 2 happen before Step 4?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 4 to ensure that the yeast is active and will properly leaven the bread.\"\n}", "relevant_step_nos": [2, 4]}
{"plan_idx": 56, "title": "white bread", "question_idx": 15, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 8?", "why_question": "Why must Step 2 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 2 happen before Step 8?\n2. Why must Step 2 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 8 because allowing the yeast to proof is necessary for the dough to rise properly.\"\n}", "relevant_step_nos": [2, 8]}
{"plan_idx": 56, "title": "white bread", "question_idx": 16, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 12?", "why_question": "Why must Step 2 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 2 happen before Step 12?\n2. Why must Step 2 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 12 because allowing the yeast to proof creates the necessary conditions for the dough to rise during the second proofing.\" \n}", "relevant_step_nos": [2, 12]}
{"plan_idx": 56, "title": "white bread", "question_idx": 17, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 2 happen before Step 9?", "why_question": "Why must Step 2 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 2 happen before Step 9?\n2. Why must Step 2 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 2 must happen before Step 9 because allowing the yeast to proof is necessary for the dough to rise properly.\"\n}", "relevant_step_nos": [2, 9]}
{"plan_idx": 56, "title": "white bread", "question_idx": 18, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 7?", "why_question": "Why must Step 1 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 1 happen before Step 7?\n2. Why must Step 1 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to activate the yeast and start the rising process.\"\n}", "relevant_step_nos": [1, 7]}
{"plan_idx": 56, "title": "white bread", "question_idx": 19, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 1 happen before Step 13?", "why_question": "Why must Step 1 happen before Step 13?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 1 happen before Step 13?\n2. Why must Step 1 happen before Step 13?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 1 is necessary to activate the yeast before baking the bread in Step 13.\"\n}", "relevant_step_nos": [1, 13]}
{"plan_idx": 56, "title": "white bread", "question_idx": 20, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 10?", "why_question": "Why must Step 8 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 8 happen before Step 10?\n2. Why must Step 8 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 allows the dough to rise and double in bulk, making it easier to knead and divide in half in Step 10.\"\n}", "relevant_step_nos": [8, 10]}
{"plan_idx": 56, "title": "white bread", "question_idx": 21, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 8 happen before Step 9?", "why_question": "Why must Step 8 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 8 happen before Step 9?\n2. Why must Step 8 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 8 allows the dough to rise and double in bulk, making it easier to work with in Step 9.\"\n}", "relevant_step_nos": [8, 9]}
{"plan_idx": 56, "title": "white bread", "question_idx": 22, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 6?", "why_question": "Why must Step 4 happen before Step 6?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 4 happen before Step 6?\n2. Why must Step 4 happen before Step 6?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 6 because the flour needs to be gradually mixed into the yeast mixture to form the dough before it is kneaded and placed in the oiled bowl.\"\n}", "relevant_step_nos": [4, 6]}
{"plan_idx": 56, "title": "white bread", "question_idx": 23, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 12?", "why_question": "Why must Step 4 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 4 happen before Step 12?\n2. Why must Step 4 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 12 because the dough needs to rise before it can be shaped into loaves and allowed to rise again.\"\n}", "relevant_step_nos": [4, 12]}
{"plan_idx": 56, "title": "white bread", "question_idx": 24, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 7?", "why_question": "Why must Step 4 happen before Step 7?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 4 happen before Step 7?\n2. Why must Step 4 happen before Step 7?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 7 to allow the dough to rise properly.\"\n}", "relevant_step_nos": [4, 7]}
{"plan_idx": 56, "title": "white bread", "question_idx": 25, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 8?", "why_question": "Why must Step 4 happen before Step 8?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 4 happen before Step 8?\n2. Why must Step 4 happen before Step 8?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 8 because the dough needs to rise, and the flour is added before the rising process.\" \n}", "relevant_step_nos": [4, 8]}
{"plan_idx": 56, "title": "white bread", "question_idx": 26, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 10?", "why_question": "Why must Step 4 happen before Step 10?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 4 happen before Step 10?\n2. Why must Step 4 happen before Step 10?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 10 because the dough needs to be properly kneaded and mixed before it can be divided and shaped into loaves.\"\n}", "relevant_step_nos": [4, 10]}
{"plan_idx": 56, "title": "white bread", "question_idx": 27, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 4 happen before Step 9?", "why_question": "Why must Step 4 happen before Step 9?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 4 happen before Step 9?\n2. Why must Step 4 happen before Step 9?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 4 must happen before Step 9 because kneading the dough after adding all the flour is necessary to develop the gluten and create the desired texture for the bread.\"\n}", "relevant_step_nos": [4, 9]}
{"plan_idx": 56, "title": "white bread", "question_idx": 28, "steps": ["In a large bowl, dissolve the sugar in warm water, and then stir in yeast.", "Allow to proof until yeast resembles a creamy foam.", "Mix salt and oil into the yeast mixture.", "Mix in flour about 125g at a time.", "Knead dough on a lightly floured surface until smooth.", "Place in a well oiled bowl, and turn dough to coat.", "Cover with a damp cloth.", "Allow to rise until doubled in bulk, about 1 hour.", "Punch dough down.", "Knead for a few minutes, and divide in half.", "Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.", "Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.", "Bake in a preheated 180 C / Gas 4 oven for 30 minutes."], "question_type": "dependent_real_before", "binary_question": "Must Step 10 happen before Step 12?", "why_question": "Why must Step 10 happen before Step 12?", "model_name": "gpt-3.5-turbo-1106", "prompt_key": "basic_binary_must_why", "model_input": "Given a goal, a procedure to achieve that goal and a question about the steps in the procedure, you are required to answer the questions.\n\nGoal: white bread\n\nProcedure:\n1. In a large bowl, dissolve the sugar in warm water, and then stir in yeast.\n2. Allow to proof until yeast resembles a creamy foam.\n3. Mix salt and oil into the yeast mixture.\n4. Mix in flour about 125g at a time.\n5. Knead dough on a lightly floured surface until smooth.\n6. Place in a well oiled bowl, and turn dough to coat.\n7. Cover with a damp cloth.\n8. Allow to rise until doubled in bulk, about 1 hour.\n9. Punch dough down.\n10. Knead for a few minutes, and divide in half.\n11. Shape into loaves, and place into two well oiled 23x13cm (9x5 in) loaf tins.\n12. Allow to rise for 30 minutes, or until dough has risen 2.5cm (1 in) above tins.\n13. Bake in a preheated 180 C / Gas 4 oven for 30 minutes.\n\n\n1. Must Step 10 happen before Step 12?\n2. Why must Step 10 happen before Step 12?\n\nFormat your answer as JSON with the key value pairs \"binary_answer\": \"yes/no answer to Q1\", \"why_answer\": \"one sentence answer to Q2\"", "model_answer": "{\n  \"binary_answer\": \"yes\",\n  \"why_answer\": \"Step 10 shapes the dough into loaves, which need to rise before baking in Step 12.\"\n}", "relevant_step_nos": [10, 12]}
