Skip to content

[SoundCloud] Use long integers (64-bit) for track IDs to prevent overflows #1450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
TobiGr merged 4 commits into from
Feb 21, 2026
Merged

[SoundCloud] Use long integers (64-bit) for track IDs to prevent overflows #1450

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5d3eaca
Use 64-bit SoundCloud track IDs
G-flat Jan 28, 2026
fe9c57c
Update mocks to fix failing SoundcloudPlaylistExtractor tests
G-flat Jan 31, 2026
8a4aab4
Add SoundcloudStreamExtractor tests for a small track ID and a track …
G-flat Jan 31, 2026
df10cc1
Record mocks correctly
TobiGr Feb 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions .../schabi/newpipe/extractor/services/soundcloud/extractors/SoundcloudPlaylistExtractor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,9 +149,7 @@ public InfoItemsPage<StreamInfoItem> getInitialPage() {
streamInfoItemsCollector.commit(
new SoundcloudStreamInfoItemExtractor(track));
} else {
// %09d would be enough, but a 0 before the number does not create
// problems, so let's be sure
ids.add(String.format("%010d", track.getInt("id")));
Copy link
Copy Markdown
Contributor

@TobiGr TobiGr Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change causes the mocks to be invalid. Please add a test for a very small ID and also for one that is exceeding the 32 bit of an int. You can fix the tests by recording the mocks for all SoundCloud tests again (see our docs for more info).

Copy link
Copy Markdown
Contributor Author

@G-flat G-flat Jan 31, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the SoundcloudPlaylistExtractor mocks and also added two SoundcloudStreamExtractor tests, one for a small track ID (135535700) and one for a track ID that's larger than 32 bits of an int (2167944333).

ids.add(String.valueOf(track.getLong("id")));
}
});

Expand Down Expand Up @@ -187,15 +185,15 @@ public InfoItemsPage<StreamInfoItem> getPage(final Page page) throws IOException
final JsonArray tracks = JsonParser.array().from(response);
// Response may not contain tracks in the same order as currentIds.
// The streams are displayed in the order which is used in currentIds on SoundCloud.
final HashMap<Integer, JsonObject> idToTrack = new HashMap<>();
final HashMap<Long, JsonObject> idToTrack = new HashMap<>();
for (final Object track : tracks) {
if (track instanceof JsonObject) {
final JsonObject o = (JsonObject) track;
idToTrack.put(o.getInt("id"), o);
idToTrack.put(o.getLong("id"), o);
}
}
for (final String strId : currentIds) {
final int id = Integer.parseInt(strId);
final long id = Long.parseLong(strId);
try {
collector.commit(new SoundcloudStreamInfoItemExtractor(
Objects.requireNonNull(
Expand Down
2 changes: 1 addition & 1 deletion ...rg/schabi/newpipe/extractor/services/soundcloud/extractors/SoundcloudStreamExtractor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ public void onFetchPage(@Nonnull final Downloader downloader) throws IOException
@Nonnull
@Override
public String getId() {
return String.valueOf(track.getInt("id"));
return String.valueOf(track.getLong("id"));
}

@Nonnull
Expand Down
76 changes: 76 additions & 0 deletions .../java/org/schabi/newpipe/extractor/services/soundcloud/SoundcloudStreamExtractorTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,4 +217,80 @@ public void testAudioStreams() throws Exception {
});
}
}

@Nested
class SmallTrackId extends DefaultStreamExtractorTest {
private static final String ID = "technicality";
private static final String UPLOADER = SOUNDCLOUD + "garrett-cameron-1";
private static final int TIMESTAMP = 34;
private static final String URL = UPLOADER + "/" + ID + "#t=" + TIMESTAMP;

@Override
protected StreamExtractor createExtractor() throws Exception {
return SoundCloud.getStreamExtractor(URL);
}

@Override public StreamingService expectedService() { return SoundCloud; }
@Override public String expectedName() { return "Technicality"; }
@Override public String expectedId() { return "135535700"; }
@Override public String expectedUrlContains() { return UPLOADER + "/" + ID; }
@Override public String expectedOriginalUrlContains() { return URL; }

@Override public StreamType expectedStreamType() { return StreamType.AUDIO_STREAM; }
@Override public String expectedUploaderName() { return "Samuel Cameron (ZVW)"; }
@Override public String expectedUploaderUrl() { return UPLOADER; }
@Override public boolean expectedDescriptionIsEmpty() { return true; }
@Override public List<String> expectedDescriptionContains() { return Collections.emptyList(); }
@Override public long expectedLength() { return 182; }
@Override public long expectedTimestamp() { return TIMESTAMP; }
@Override public long expectedViewCountAtLeast() { return 16; }
@Nullable @Override public String expectedUploadDate() { return "2014-02-18 21:25:01.000"; }
@Nullable @Override public String expectedTextualUploadDate() { return "2014-02-18T21:25:01Z"; }
@Override public long expectedLikeCountAtLeast() { return 1; }
@Override public long expectedDislikeCountAtLeast() { return -1; }
@Override public boolean expectedHasVideoStreams() { return false; }
@Override public boolean expectedHasSubtitles() { return false; }
@Override public boolean expectedHasFrames() { return false; }
@Override public int expectedStreamSegmentsCount() { return 0; }
@Override public String expectedLicence() { return "all-rights-reserved"; }
@Override public String expectedCategory() { return "Electronic"; }
}

@Nested
class LongTrackId extends DefaultStreamExtractorTest {
private static final String ID = "hard-to-break";
private static final String UPLOADER = SOUNDCLOUD + "blakewhiten";
private static final int TIMESTAMP = 130;
private static final String URL = UPLOADER + "/" + ID + "#t=" + TIMESTAMP;

@Override
protected StreamExtractor createExtractor() throws Exception {
return SoundCloud.getStreamExtractor(URL);
}

@Override public StreamingService expectedService() { return SoundCloud; }
@Override public String expectedName() { return "Hard to Break"; }
@Override public String expectedId() { return "2167944333"; }
@Override public String expectedUrlContains() { return UPLOADER + "/" + ID; }
@Override public String expectedOriginalUrlContains() { return URL; }

@Override public StreamType expectedStreamType() { return StreamType.AUDIO_STREAM; }
@Override public String expectedUploaderName() { return "Blake Whiten"; }
@Override public String expectedUploaderUrl() { return UPLOADER; }
@Override public boolean expectedDescriptionIsEmpty() { return true; }
@Override public List<String> expectedDescriptionContains() { return Collections.emptyList(); }
@Override public long expectedLength() { return 201; }
@Override public long expectedTimestamp() { return TIMESTAMP; }
@Override public long expectedViewCountAtLeast() { return 120222; }
@Nullable @Override public String expectedUploadDate() { return "2025-09-10 02:24:00.000"; }
@Nullable @Override public String expectedTextualUploadDate() { return "2025-09-10T02:24:00Z"; }
@Override public long expectedLikeCountAtLeast() { return 1970; }
@Override public long expectedDislikeCountAtLeast() { return -1; }
@Override public boolean expectedHasVideoStreams() { return false; }
@Override public boolean expectedHasSubtitles() { return false; }
@Override public boolean expectedHasFrames() { return false; }
@Override public int expectedStreamSegmentsCount() { return 0; }
@Override public String expectedLicence() { return "all-rights-reserved"; }
@Override public String expectedCategory() { return "Country"; }
}
}
14 changes: 7 additions & 7 deletions .../extractor/services/soundcloud/search/soundcloudsearchextractor/all/generated_mock_0.json
View file
Open in desktop

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions .../extractor/services/soundcloud/search/soundcloudsearchextractor/all/generated_mock_1.json
View file
Open in desktop

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions ...tor/services/soundcloud/search/soundcloudsearchextractor/nonextpage/generated_mock_0.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"request": {
"httpMethod": "GET",
"url": "https://api-v2.soundcloud.com/search?q\u003dwpgh%C3%BC%C3%A4\u0026client_id\u003dMoLbAg35TuqjYwWVtNIKyRPFScQGMOBY\u0026limit\u003d10\u0026offset\u003d0",
"url": "https://api-v2.soundcloud.com/search?q\u003dwpgh%C3%BC%C3%A4\u0026client_id\u003dCkCiIyf14rHi27fhk7HxhPOzc85okfSJ\u0026limit\u003d10\u0026offset\u003d0",
"headers": {
"Accept-Language": [
"en-GB, en;q\u003d0.9"
Expand All @@ -26,7 +26,7 @@
"application/json; charset\u003dutf-8"
],
"date": [
"Sat, 12 Jul 2025 10:24:21 GMT"
"Sat, 21 Feb 2026 01:08:32 GMT"
],
"referrer-policy": [
"no-referrer"
Expand All @@ -41,13 +41,13 @@
"Origin"
],
"via": [
"1.1 cbad29402e4e90baabe7151c3f1203b6.cloudfront.net (CloudFront)"
"1.1 cb2b840821313909da7f7890ec5606ba.cloudfront.net (CloudFront)"
],
"x-amz-cf-id": [
"dzL6w9UaEREDBeOHXXNDtPTsdqPoQdIeDb3l5Q_QtN6jqFBEXG9MhQ\u003d\u003d"
"Vm55G14YlC-_pq2Ll2GMbvwX-uRz904SxMTr5Tu4d-zCo3A6RTTiRw\u003d\u003d"
],
"x-amz-cf-pop": [
"FRA56-P11"
"HAM50-P5"
],
"x-cache": [
"Miss from cloudfront"
Expand All @@ -62,7 +62,7 @@
"noindex"
]
},
"responseBody": "{\"collection\":[{\"avatar_url\":\"https://i1.sndcdn.com/avatars-PU8AmxubPTAMnl0t-zad7tA-large.jpg\",\"city\":null,\"comments_count\":0,\"country_code\":null,\"created_at\":\"2022-02-28T12:42:23Z\",\"creator_subscriptions\":[{\"product\":{\"id\":\"free\"}}],\"creator_subscription\":{\"product\":{\"id\":\"free\"}},\"description\":null,\"followers_count\":0,\"followings_count\":0,\"first_name\":\"Weghua\",\"full_name\":\"Weghua Habibu\",\"groups_count\":0,\"id\":1091242897,\"kind\":\"user\",\"last_modified\":\"2022-02-28T12:42:35Z\",\"last_name\":\"Habibu\",\"likes_count\":0,\"playlist_likes_count\":0,\"permalink\":\"weghua-habibu\",\"permalink_url\":\"https://soundcloud.com/weghua-habibu\",\"playlist_count\":0,\"reposts_count\":null,\"track_count\":0,\"uri\":\"https://api.soundcloud.com/users/1091242897\",\"urn\":\"soundcloud:users:1091242897\",\"username\":\"Weghua Habibu\",\"verified\":false,\"visuals\":null,\"badges\":{\"pro\":false,\"creator_mid_tier\":false,\"pro_unlimited\":false,\"verified\":false},\"station_urn\":\"soundcloud:system-playlists:artist-stations:1091242897\",\"station_permalink\":\"artist-stations:1091242897\",\"date_of_birth\":{\"month\":2,\"year\":1973,\"day\":1}},{\"avatar_url\":\"https://i1.sndcdn.com/avatars-000360777944-r6mq9y-large.jpg\",\"city\":null,\"comments_count\":0,\"country_code\":null,\"created_at\":\"2017-12-05T08:50:01Z\",\"creator_subscriptions\":[{\"product\":{\"id\":\"free\"}}],\"creator_subscription\":{\"product\":{\"id\":\"free\"}},\"description\":null,\"followers_count\":0,\"followings_count\":0,\"first_name\":\"Wpg\",\"full_name\":\"Wpg Ua\",\"groups_count\":0,\"id\":360091154,\"kind\":\"user\",\"last_modified\":\"2017-12-05T08:50:02Z\",\"last_name\":\"Ua\",\"likes_count\":0,\"playlist_likes_count\":0,\"permalink\":\"wpg-ua\",\"permalink_url\":\"https://soundcloud.com/wpg-ua\",\"playlist_count\":0,\"reposts_count\":null,\"track_count\":0,\"uri\":\"https://api.soundcloud.com/users/360091154\",\"urn\":\"soundcloud:users:360091154\",\"username\":\"Wpg Ua\",\"verified\":false,\"visuals\":null,\"badges\":{\"pro\":false,\"creator_mid_tier\":false,\"pro_unlimited\":false,\"verified\":false},\"station_urn\":\"soundcloud:system-playlists:artist-stations:360091154\",\"station_permalink\":\"artist-stations:360091154\",\"date_of_birth\":null},{\"avatar_url\":\"https://a1.sndcdn.com/images/default_avatar_large.png\",\"city\":\"\",\"comments_count\":0,\"country_code\":\"CA\",\"created_at\":\"2017-08-05T22:17:50Z\",\"creator_subscriptions\":[{\"product\":{\"id\":\"free\"}}],\"creator_subscription\":{\"product\":{\"id\":\"free\"}},\"description\":\"\",\"followers_count\":0,\"followings_count\":1,\"first_name\":\"Douglas\",\"full_name\":\"Douglas Chua\",\"groups_count\":0,\"id\":324448453,\"kind\":\"user\",\"last_modified\":\"2017-08-05T22:18:05Z\",\"last_name\":\"Chua\",\"likes_count\":0,\"playlist_likes_count\":0,\"permalink\":\"douglaswpchua\",\"permalink_url\":\"https://soundcloud.com/douglaswpchua\",\"playlist_count\":0,\"reposts_count\":null,\"track_count\":0,\"uri\":\"https://api.soundcloud.com/users/324448453\",\"urn\":\"soundcloud:users:324448453\",\"username\":\"DouglasWPChua\",\"verified\":false,\"visuals\":null,\"badges\":{\"pro\":false,\"creator_mid_tier\":false,\"pro_unlimited\":false,\"verified\":false},\"station_urn\":\"soundcloud:system-playlists:artist-stations:324448453\",\"station_permalink\":\"artist-stations:324448453\",\"date_of_birth\":{\"month\":5,\"year\":1976,\"day\":1}}],\"total_results\":3,\"query_urn\":\"soundcloud:search:147d8013fd614c08a31215017d95cf50\"}",
"latestUrl": "https://api-v2.soundcloud.com/search?q\u003dwpgh%C3%BC%C3%A4\u0026client_id\u003dMoLbAg35TuqjYwWVtNIKyRPFScQGMOBY\u0026limit\u003d10\u0026offset\u003d0"
"responseBody": "{\"collection\":[{\"avatar_url\":\"https://i1.sndcdn.com/avatars-000360777944-r6mq9y-large.jpg\",\"city\":null,\"comments_count\":0,\"country_code\":null,\"created_at\":\"2017-12-05T08:50:01Z\",\"creator_subscriptions\":[{\"product\":{\"id\":\"free\"}}],\"creator_subscription\":{\"product\":{\"id\":\"free\"}},\"description\":null,\"followers_count\":0,\"followings_count\":0,\"first_name\":\"Wpg\",\"full_name\":\"Wpg Ua\",\"groups_count\":0,\"id\":360091154,\"kind\":\"user\",\"last_modified\":\"2017-12-05T08:50:02Z\",\"last_name\":\"Ua\",\"likes_count\":0,\"playlist_likes_count\":0,\"permalink\":\"wpg-ua\",\"permalink_url\":\"https://soundcloud.com/wpg-ua\",\"playlist_count\":0,\"reposts_count\":null,\"track_count\":0,\"uri\":\"https://api.soundcloud.com/users/soundcloud%3Ausers%3A360091154\",\"urn\":\"soundcloud:users:360091154\",\"username\":\"Wpg Ua\",\"verified\":false,\"visuals\":null,\"badges\":{\"pro\":false,\"creator_mid_tier\":false,\"pro_unlimited\":false,\"verified\":false},\"station_urn\":\"soundcloud:system-playlists:artist-stations:360091154\",\"station_permalink\":\"artist-stations:360091154\",\"date_of_birth\":null},{\"avatar_url\":\"https://i1.sndcdn.com/avatars-PU8AmxubPTAMnl0t-zad7tA-large.jpg\",\"city\":null,\"comments_count\":0,\"country_code\":null,\"created_at\":\"2022-02-28T12:42:23Z\",\"creator_subscriptions\":[{\"product\":{\"id\":\"free\"}}],\"creator_subscription\":{\"product\":{\"id\":\"free\"}},\"description\":null,\"followers_count\":0,\"followings_count\":0,\"first_name\":\"Weghua\",\"full_name\":\"Weghua Habibu\",\"groups_count\":0,\"id\":1091242897,\"kind\":\"user\",\"last_modified\":\"2022-02-28T12:42:35Z\",\"last_name\":\"Habibu\",\"likes_count\":0,\"playlist_likes_count\":0,\"permalink\":\"weghua-habibu\",\"permalink_url\":\"https://soundcloud.com/weghua-habibu\",\"playlist_count\":0,\"reposts_count\":null,\"track_count\":0,\"uri\":\"https://api.soundcloud.com/users/soundcloud%3Ausers%3A1091242897\",\"urn\":\"soundcloud:users:1091242897\",\"username\":\"Weghua Habibu\",\"verified\":false,\"visuals\":null,\"badges\":{\"pro\":false,\"creator_mid_tier\":false,\"pro_unlimited\":false,\"verified\":false},\"station_urn\":\"soundcloud:system-playlists:artist-stations:1091242897\",\"station_permalink\":\"artist-stations:1091242897\",\"date_of_birth\":null},{\"avatar_url\":\"https://a1.sndcdn.com/images/default_avatar_large.png\",\"city\":\"\",\"comments_count\":0,\"country_code\":\"CA\",\"created_at\":\"2017-08-05T22:17:50Z\",\"creator_subscriptions\":[{\"product\":{\"id\":\"free\"}}],\"creator_subscription\":{\"product\":{\"id\":\"free\"}},\"description\":\"\",\"followers_count\":0,\"followings_count\":1,\"first_name\":\"Douglas\",\"full_name\":\"Douglas Chua\",\"groups_count\":0,\"id\":324448453,\"kind\":\"user\",\"last_modified\":\"2017-08-05T22:18:05Z\",\"last_name\":\"Chua\",\"likes_count\":0,\"playlist_likes_count\":0,\"permalink\":\"douglaswpchua\",\"permalink_url\":\"https://soundcloud.com/douglaswpchua\",\"playlist_count\":0,\"reposts_count\":null,\"track_count\":0,\"uri\":\"https://api.soundcloud.com/users/soundcloud%3Ausers%3A324448453\",\"urn\":\"soundcloud:users:324448453\",\"username\":\"DouglasWPChua\",\"verified\":false,\"visuals\":null,\"badges\":{\"pro\":false,\"creator_mid_tier\":false,\"pro_unlimited\":false,\"verified\":false},\"station_urn\":\"soundcloud:system-playlists:artist-stations:324448453\",\"station_permalink\":\"artist-stations:324448453\",\"date_of_birth\":null}],\"total_results\":3,\"query_urn\":\"soundcloud:search:621caed9c6f24445bee3d04a783338aa\"}",
"latestUrl": "https://api-v2.soundcloud.com/search?q\u003dwpgh%C3%BC%C3%A4\u0026client_id\u003dCkCiIyf14rHi27fhk7HxhPOzc85okfSJ\u0026limit\u003d10\u0026offset\u003d0"
}
}
14 changes: 7 additions & 7 deletions ...tractor/services/soundcloud/search/soundcloudsearchextractor/paging/generated_mock_0.json
View file
Open in desktop

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions ...tractor/services/soundcloud/search/soundcloudsearchextractor/paging/generated_mock_1.json
View file
Open in desktop

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions ...ctor/services/soundcloud/search/soundcloudsearchextractor/playlists/generated_mock_0.json
View file
Open in desktop

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions ...ctor/services/soundcloud/search/soundcloudsearchextractor/playlists/generated_mock_1.json
View file
Open in desktop

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions ...tractor/services/soundcloud/search/soundcloudsearchextractor/tracks/generated_mock_0.json
View file
Open in desktop

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions ...tractor/services/soundcloud/search/soundcloudsearchextractor/tracks/generated_mock_1.json
View file
Open in desktop

Large diffs are not rendered by default.

Loading