Skip to content

Conversation

@cmaglie
Copy link
Member

@cmaglie cmaglie commented Oct 3, 2025

Please check if the PR fulfills these requirements

See how to contribute

  • The PR has no duplicates (please search among the Pull Requests
    before creating one)
  • The PR follows
    our contributing guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • UPGRADING.md has been updated with a migration guide (for breaking changes)
  • configuration.schema.json updated if new parameters are added.

What kind of change does this PR introduce?

This is an extract of #2917 containing only the gRPC part of the PR.

What is the current behavior?

Adds only the gRPC functions for managing libraries in profiles, no user-facing changes.

What is the new behavior?

Does this PR introduce a breaking change, and is titled accordingly?

No

Other information

@cmaglie cmaglie self-assigned this Oct 3, 2025
@cmaglie cmaglie added type: enhancement Proposed improvement topic: gRPC Related to the gRPC interface labels Oct 3, 2025
@per1234 per1234 added the topic: code Related to content of the project itself label Oct 4, 2025
per1234
per1234 previously requested changes Oct 5, 2025
@codecov
Copy link

codecov bot commented Oct 6, 2025

Codecov Report

❌ Patch coverage is 18.41004% with 390 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.34%. Comparing base (a8423ab) to head (0c28634).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
commands/service_profile_lib_add.go 0.00% 104 Missing ⚠️
internal/arduino/sketch/profiles.go 18.03% 91 Missing and 9 partials ⚠️
commands/service_profile_init.go 0.00% 59 Missing ⚠️
commands/service_profile_lib_remove.go 0.00% 33 Missing ⚠️
commands/instances.go 0.00% 25 Missing and 1 partial ⚠️
commands/service_profile_lib_list.go 0.00% 20 Missing ⚠️
commands/service_profile_set_default.go 0.00% 19 Missing ⚠️
commands/cmderrors/cmderrors.go 0.00% 10 Missing and 1 partial ⚠️
...rnal/arduino/libraries/librariesmanager/install.go 65.21% 7 Missing and 1 partial ⚠️
commands/service_library_install.go 56.25% 5 Missing and 2 partials ⚠️
... and 1 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3019      +/-   ##
==========================================
- Coverage   68.39%   67.34%   -1.06%     
==========================================
  Files         241      246       +5     
  Lines       22734    23137     +403     
==========================================
+ Hits        15550    15582      +32     
- Misses       5988     6350     +362     
- Partials     1196     1205       +9     
Flag Coverage Δ
unit 67.34% <18.41%> (-1.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

As stated by @per1234:

> Build profile data is already provided via the LoadSketch method, so it
> seems that even a mechanism that is truly for getting profile data should
> be implemented by simply expanding the SketchProfile message to contain
> all the data of the build profile (actually kind of silly that it
> currently only provides a subset of the profile data).

arduino#3019 (comment)
@per1234 per1234 dismissed their stale review October 6, 2025 13:17

Requested change has been made. Thanks!

// Name of the library.
string name = 1;
// Version of the library if taken from the Library Index.
string version = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clients would love to use a dedicated message to add the latest version of the library (without listing the available versions), and maybe add the currently installed version that errors if the library is not installed.

@cmaglie cmaglie force-pushed the grpc-profiles-functions branch from dbac430 to 2327b35 Compare October 29, 2025 10:00
@dankeboy36
Copy link
Contributor

Hello Cristian,

Thanks for your great work 💪 I’ve been testing this area from the IDE side and was wondering how this feature is expected to work with an IDE or editor. Maybe this isn’t the most appropriate channel, but I guess you’re the best person to ask.

It’s clear there’s a real need for this feature:

While reading the PR and the related IDE issue arduino/arduino-ide#2573, I was trying to understand how profiles are meant to behave over gRPC, maybe later in Arduino IDE 2.x:

  • If an IDE opens two sketches with different profiles (e.g. arduino:avr:uno and esp32:esp32:esp32da), should each sketch have its own gRPC client or session?
  • When switching or activating a profile, does the CLI reinitialize the build environment (toolchains, platforms, libraries), or are these shared and cached? For example, if libA is installed in profileA but not in profileB, will a library search over gRPC return different results for each profile?
  • If multiple profiles use the same library, will it be installed multiple times, or can they share a global copy? Since each sketch can have its own sketch.yaml, does that mean the same dependency could be downloaded many times?
  • When using gRPC, should compile, upload, etc. always load fqbn, port, and other settings from the active profile, or should clients omit them?

I really like how profiles make builds reproducible and easy to share — that’s perfect for CI or collaborative work. But in an IDE that uses gRPC interactively, managing multiple isolated clients could get heavy, especially if it means repeated downloads.

In my current setup I treat sketch.yaml mostly as a lightweight config file. Libraries and platforms are managed globally, and if something is missing, the user only gets a warning instead of full profile enforcement.

I’d be curious to hear how Arduino sees this working long-term on the gRPC side — especially for IDE integration. Will profiles act more like isolated environments or just as stored build settings that can be written and updated through gRPC?

No hurry with this. Thanks a lot for your time

@cmaglie
Copy link
Member Author

cmaglie commented Nov 17, 2025

Hi @dankeboy36!

Thank you for your valuable feedback, as always.

If an IDE opens two sketches with different profiles (e.g. arduino:avr:uno and esp32:esp32:esp32da), should each sketch have its own gRPC client or session?

The basic idea is that when a profile is in use, the platforms and libraries installed globally should have no effect, and only the platforms and libraries listed in the profile should be visible. This idea was implemented by adding the profile parameter in the Init method:

message InitRequest {
  // An Arduino Core instance.
  Instance instance = 1;
  // Profile to use.
  string profile = 2;
  // The path where the sketch is stored.
  string sketch_path = 3;
}

An instance initialized with a profile should be able to see only the libraries listed in the profile, and I've actually checked that this is the case, using the following profile:

profiles:
  uno:
    fqbn: arduino:avr:uno
    platforms:
      - platform: arduino:avr (1.8.3)
    libraries:
      - Servo (1.1.6)

If I try to initialize an instance of the CLI with the profile and try to get the list of installed libraries:

		err := srv.Init(&rpc.InitRequest{
			Instance:   inst,
			SketchPath: sketchPath.String(),
			Profile:    "uno",
		}, stream)
		res, err := srv.LibraryList(ctx, &rpc.LibraryListRequest{
			Instance: inst,
			All:      true,
		})
		require.NoError(t, err)
		for _, lib := range res.InstalledLibraries {
			fmt.Println("LIB>",
				lib.GetLibrary().GetName(),
				lib.GetLibrary().GetVersion(),
				lib.GetLibrary().GetLocation(),
				lib.GetLibrary().GetInstallDir())
		}

I got the expected:

LIB> Servo 1.1.6 LIBRARY_LOCATION_PROFILE /home/cmaglie/.arduino15/internal/Servo_1.1.6_663e17729cbd9f93/Servo
LIB> Wire 1.0 LIBRARY_LOCATION_PLATFORM_BUILTIN /home/cmaglie/.arduino15/internal/arduino_avr_1.8.3_40b0bc3206d75f60/libraries/Wire
LIB> EEPROM 2.0 LIBRARY_LOCATION_PLATFORM_BUILTIN /home/cmaglie/.arduino15/internal/arduino_avr_1.8.3_40b0bc3206d75f60/libraries/EEPROM
LIB> HID 1.0 LIBRARY_LOCATION_PLATFORM_BUILTIN /home/cmaglie/.arduino15/internal/arduino_avr_1.8.3_40b0bc3206d75f60/libraries/HID
LIB> SPI 1.0 LIBRARY_LOCATION_PLATFORM_BUILTIN /home/cmaglie/.arduino15/internal/arduino_avr_1.8.3_40b0bc3206d75f60/libraries/SPI
LIB> SoftwareSerial 1.0 LIBRARY_LOCATION_PLATFORM_BUILTIN /home/cmaglie/.arduino15/internal/arduino_avr_1.8.3_40b0bc3206d75f60/libraries/SoftwareSerial

The Servo library is listed as LOCATION_PROFILE, the other libraries are PLATFORM_BUILTIN.

So, based on the previous test, the answer to the following question is:

When switching or activating a profile, does the CLI reinitialize the build environment (toolchains, platforms, libraries), or are these shared and cached? For example, if libA is installed in profileA but not in profileB, will a library search over gRPC return different results for each profile?

Yes, the result will differ depending on the profile. By the way, there is a distinction between a library search (i.e., searching for a library in the Library Index) and listing installed libraries. In the first case, searching for a library will always produce the same result regardless of the selected profile; however, listing the installed libraries will show different results based on the selected profile.

If multiple profiles use the same library, will it be installed multiple times, or can they share a global copy? Since each sketch can have its own sketch.yaml, does that mean the same dependency could be downloaded many times?

Libraries and platforms installed through profiles are placed in a private folder inside $DATA_DIR/internal/. The same package will not be downloaded twice; there are no duplicates here. Those directories should be read-only. By the way, there are no protections in place to prevent users from making changes.

$ ls ~/.arduino15/internal/ -l
totale 200
drwxr-xr-x  3 cmaglie cmaglie 4096 ott  3  2024  Adafruit_BusIO_1.16.1_ba866f2a3757c639
drwxr-xr-x  4 cmaglie cmaglie 4096 ott  9  2024 'Adafruit SleepyDog Library@1.6.5_a3c453d34b90db4b'
drwxrwxr-x  3 cmaglie cmaglie 4096 mag 30 16:02  arduino_arduinoOTA_1.2.1_c939277d7d79c18d
drwxr-xr-x  3 cmaglie cmaglie 4096 gen 23  2024  arduino_arduinoOTA_1.3.0_8902221f9b4869ce
drwxr-xr-x  6 cmaglie cmaglie 4096 mag 30 16:02  arduino_arm-none-eabi-gcc_7-2017q4_7b7be9f526b2cb64
drwxr-xr-x  8 cmaglie cmaglie 4096 nov 17 11:55  arduino_avr_1.8.3_40b0bc3206d75f60
drwxr-xr-x  8 cmaglie cmaglie 4096 gen 23  2024  arduino_avr_1.8.4_7ad8ae8e9f6a9dc9
drwxr-xr-x  9 cmaglie cmaglie 4096 gen 23  2024  arduino_avr_1.8.5_70334d3546efee53
drwxr-xr-x  9 cmaglie cmaglie 4096 lug  1  2024  arduino_avr_1.8.6_78e0815c6047bc54
drwxr-xr-x  4 cmaglie cmaglie 4096 gen 23  2024  arduino_avrdude_6.3.0-arduino17_b00caa3114ef923d
drwxr-xr-x  7 cmaglie cmaglie 4096 mag 30 16:03  arduino_avrdude_6.3.0-arduino9_734fb0aeab0d748f
drwxr-xr-x  8 cmaglie cmaglie 4096 gen 23  2024  arduino_avr-gcc_7.3.0-atmel3.6.1-arduino7_fc423089e09cfcab
drwxr-xr-x  3 cmaglie cmaglie 4096 lug  7 17:10  ArduinoBearSSL_1.7.6_3fd6f33f302ce290

When using gRPC, should compile, upload, etc. always load fqbn, port, and other settings from the active profile, or should clients omit them?

The Compile and Upload commands should retrieve the FQBN and Port from the profile, and the gRPC API allows overriding them if the FQBN and Port parameters are filled. At least, that was the intention.
I put the above statement to the test, and I discovered that:

  • Compile always requires the sketch path (even if it was already specified in the Init call)
  • Upload always requires the sketch path (even if it was already specified in the Init call)
  • Upload always requires the port address (even if it was present in the profile)
    The CLI UI actually covers this behaviour by filling in the missing pieces in the gRPC calls.

The above must surely be fixed in the next releases. I'll open an issue for that.

@cmaglie
Copy link
Member Author

cmaglie commented Nov 17, 2025

We are currently using this API in the AppLab IDE to manage profiles in the sketch. This is still experimental, and your suggestions and thoughts are more than welcome. Thanks for taking the time to write them down. 🙏🏼

I’d be curious to hear how Arduino sees this working long-term on the gRPC side — especially for IDE integration. Will profiles act more like isolated environments or just as stored build settings that can be written and updated through gRPC?

At the moment, I would say that they serve both purposes:

  • They provide isolated environments, even reproducible builds if all the versions and URLs are correctly specified.
  • In a certain sense, they store the build settings, too? (having all the resources needed to build a sketch).

The hard question is how all of this integrates with the IDE?

My envisioned gRPC API involves the IDE creating multiple instances (using the Create gRPC call), one for each open "sketch." Each instance should then be initialized (with the Init method), optionally with a profile, while the CLI manages the resources (possibly reusing/sharing to avoid wasting resources).
If the user selects a different profile, the instance should be re-initialized.

Currently, I understand the Arduino IDE 2 uses a global instance (without a profile) that might be shared across all sketches that do not use profiles. I see that this could complicate things when the profiles are introduced because each UX business logic needs to be implemented twice, once for the classic "non-profiled" sketches and once for "profiled" sketches.

There are still changes to the API to be made to make it IDE-friendly. The current state of the gRPC API is still not ready for that, but the above is the direction I'd like to see in the future.

@per1234 per1234 added the topic: package-management Related to the packaging and managing of the platform/libraries label Nov 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic: code Related to content of the project itself topic: gRPC Related to the gRPC interface topic: package-management Related to the packaging and managing of the platform/libraries type: enhancement Proposed improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants