The first Swordsman was Jacques DuQuesne. Along with Trickshot, he helped train the Avenger known as Hawkeye. Swordsman was caught stealing money from the circus where all three men worked as performers. Hawkeye attempted to report Swordsman to the police only for his one-time mentor to beat him half to death. Swordsman left the circus afterward and began a career as a mercenary which subsequently saw him deported and exiled from several countries where he had committed crimes.
Swordsman was introduced demanding to join the Avengers and was sharply ejected after being exposed as a traitor. Following his dismissal from Earth's Mightiest, he began working with an assortment of villains in opposing the Avengers such as Egghead, Batroc's Brigade, and the Lethal Legion. He would eventually rejoin the Avengers for the assault on Olympus and later travel around the world as a drunken mercenary until the woman known as Mantis entered his life. She helped Swordsman regain his self-respect and, with her at his side, he once again joined the ranks of the Avengers.