Neuronal primary cilia are signaling organelles with crucial roles in brain development and disease. Cilia structure is decisive for their signaling capacities but the mechanisms regulating it are poorly understood. We identify Fbxo41 as a novel Skp1/Cullin1/F-box (SCF) E3-ligase complex subunit that targets to neuronal centrioles where its accumulation promotes disassembly of primary cilia, and affects sonic hedgehog signaling, a canonical ciliary pathway. Fbxo41 targeting to centrioles requires its Coiled-coil and F-box domains. Levels of Fbxo41 at the centrioles inversely correlate with neuronal cilia length, and mutations that disrupt Fbxo41 targeting or assembly into SCF-complexes also disturb its function in cilia disassembly and signaling. Fbxo41 dependent cilia disassembly in mitotic and post-mitotic cells requires rearrangements of the actin-cytoskeleton, but requires Aurora A kinase activation only in mitotic cells, highlighting important mechanistical differences controlling cilia size between mitotic and post-mitotic cells. Phorbol esters induce recruitment of overexpressed Fbxo41 to centrioles and cilia disassembly in neurons, but disassembly can also occur in absence of Fbxo41. We propose that Fbxo41 targeting to centrosomes regulates neuronal cilia structure and signaling capacity in addition to Fbxo41-independent pathways controlling cilia size.