Previous research indicates that adding auditory effects to a (silent) action can lead to substantial efficiency gains in the performance of the action, while compromising the connection between the motor and the auditory event (e.g., by removing, or by delaying the auditory effects), leads the agent to compensate for the loss of auditory feedback by executing actions in a way which increases the probability of success or enhances feedback in other modalities, thus departing from the optimal action performance. The current study explored how this motor adaptation was affected when the quality of auditory feedback was reduced by contextual factors, while keeping the physical link between the action and auditory effect intact. In two experiments, participants elicited pure tones by pinching a force sensitive resistor (FSR). In some of the conditions action-effect contingency was reduced by intermixing externally initiated tones with the self-induced ones. Pinch-force measurements indicated that action optimization was affected by contextual factors. The influence of auditory context was the most pronounced when the discrimination of self-induced and external tones was made difficult by the similarity and temporal proximity of the self-induced and external tones. In these conditions, tone eliciting actions were more forceful in comparison to conditions in which no external tones were presented, and in comparison to conditions in which the external tones were easily distinguishable from self-induced ones. This suggests that contextual factors can induce similar motor adjustments as manipulating the physical connection between the action and its sensory consequences.