Preheat the oven to 350 degrees F (175 degrees C).
Combine sugar, flour, and salt in a saucepan. Gradually stir in milk. Cook, stirring constantly, over medium heat until mixture boils and thickens. Continue to stir and cook for 2 more minutes, then remove from the burner.
Place egg yolks in a medium bowl. Whisk in a small amount of hot mixture, then immediately add egg yolk mixture to remaining hot mixture.
Return the saucepan to the burner and cook, stirring constantly, for 2 more minutes. Remove the saucepan from the stove; stir in butter and vanilla.
Fill pastry shell with sliced bananas. Cover with pudding.