Currently we are using GIT as source control for SSIS packages. We use package deployment model with each package has a relevant .dtsConfig File like the below.
For example,
Package1.dtsx
Config1.dtsConfig
We have many environments like Dev, Dev-1, STG, STG-1,........ We do SSIS package deployments (File System Deployment) for all the environments manually through DBA assistance.
We planned to automate SSIS package deployments through Devops. In order to do that, we have to create a generic folder structure like the below.
Packages
Package1.dtsx
Dev
Config1.dtsConfig
Dev-1
Config1.dtsConfig
STG
Config1.dtsConfig
STG-1
Config1.dtsConfig
....
....
I mean the package.dtsx file is common for all environments so we place it in Packages folder. Created a separate folder for each environment to place config files.
Since configuration details are changeable based on environment.
I am new to SSIS deployments, I am not sure is this feasible or not.
Is any way we can plan to do the deployments through Devops. What would be the best idea to implement this process. Is any better way we can do it.
Yes. You can setup something you are expecting and keep the same config file. You can update the configuration values while deploying to respective environment using powershell.
Alternatively, you can consider migrating the SSIS solution to Project deployment mode to SSISDB(I Believe it is Available for 2012). Which is most preferred way.
Related
I have a solution with many projects, and we are migrating to .NET SDK style projects, but for now we have a mix of .NET framework style projects and .NET SDK style projects.
We are also migrating to GitHub actions, and this solution was building without errors previously, but the restore action started failing when dotnet was updated from 6.0.300 to 6.0.400 (update: I tried targeting 6.0.300 specifically in the setup-dotnet action, but it's throwing the same errors, so I'm not sure what changed to cause it to fail like this when it was working before.)
I updated our local action server to 6.0.400, and when I run the command dotnet restore ./path/to/solution.sln it is restoring the NuGet packages for just the .NET SDK style projects as expected.
dotnet is installed with this GitHub action
- name: Setup .NET
uses: actions/setup-dotnet#v2
with:
dotnet-version: 6.0.x
and restore is being called with this GitHub action
- name: Restore dependencies
run: dotnet restore ${{env.SOLUTION_FILE_PATH}}
and when dotnet restore is being run from the GitHub action, I am getting the following error error MSB4057: The target "Restore" does not exist in the project. for all of the .NET SDK style projects. It's as if it's trying to restore NuGet packages using the older .NET Framework style NuGet packages. This is very different than what I've seen before and is unexpected. I have a separate action for calling nuget restore ./path/to/solution.sln for restoring packages for the .NET framework style projects, and I'm expecting dotnet restore to only restore the .NET SDK style projects.
Has anyone else run into similar problems with dotnet 6.0.400? Are there better options for restoring NuGet packages in GitHub actions?
I'm not really sure where to look next because running the command line commands locally work exactly how I would expect them to, and it only behaves oddly when getting called from GitHub actions.
Update:
I've been able to reproduce the failure locally by running the dotnet version that is being installed locally as part of actions/setup-dotnet#v2
If I run dotnet restore ... from the global install location C:\Program Files\dotnet\dotnet.exe then I get the following output which is what I expect
Determining projects to restore...
Restored C:\actions-runner\_work\MySolution\MySolution\src\FirstSdkProject\FirstSdkProject.csproj (in 335 ms).
Restored C:\actions-runner\_work\MySolution\MySolution\src\SecondSdkProject\SecondSdkProject.csproj (
in 357 ms).
If I restore from the locally installed dotnet at C:\Users\MyUser\AppData\Local\Microsoft\dotnet then I get the unexpected output that I'm getting in the GitHub action
Determining projects to restore...
Determining projects to restore...
C:\actions-runner\_work\MySolution\MySolution\src\FirstSdkProject\FirstSdkProject.csproj : error MSB4057: The target "Restore" does not exist in the project.
Nothing to do. None of the projects specified contain packages to restore.
Determining projects to restore...
Nothing to do. None of the projects specified contain packages to restore.
Nothing to do. None of the projects specified contain packages to restore.
Determining projects to restore...
C:\actions-runner\_work\MySolution\MySolution\src\SecondSdkProject\SecondSdkProject.csproj : error MSB4057: The target "Restore" does not exist in the project.
Nothing to do. None of the projects specified contain packages to restore.
Nothing to do. None of the projects specified contain packages to restore.
Determining projects to restore...
Nothing to do. None of the projects specified contain packages to restore.
Nothing to do. None of the projects specified contain packages to restore.
Nothing to do. None of the projects specified contain packages to restore.
Nothing to do. None of the projects specified contain packages to restore.
Comparing the information on dotnet.exe, they are the exact same version of dotnet and using a file compare program they are binary the same, and the folders they are in are seemingly the same as well with only a few minor differences. Why would running restore have two very different outcomes just running from different locations?
One of the main differences between a .NET Framework project and a .NET SDK project is how NuGet package references are managed. With .NET Framework projects, NuGet references are managed in packages.config. To build and restore including packages.config references you need the following
msbuild -t:build -restore -p:RestorePackagesConfig=true ./path/to/solution.sln
If you don't have mixed projects and they all follow the .NET SDK csproj format, then you won't have any packages.config references and you can build and restore with this.
msbuild -t:build -restore ./path/to/solution.sln
The option RestorePackagesConfig is only available on msbuild 16.5+ https://learn.microsoft.com/en-us/nuget/reference/msbuild-targets#restoring-packagereference-and-packagesconfig-projects-with-msbuild
I have a web (online calculator for an example) which developed by my fellow tem members. Now they want to deploy in PCF using manifests.
Languages used : python, php and javascipt.
I gone through the docs about pcf with manifest.yml
In that I don't have any idea about services and env.
What is that services and how can I find the services for the above project and also how can I find the environment variables?
And tell whether these fields are mandatory to run the project in PCF.
To your original question:
What is that services and how can I find the services for the above project and also how can I find the environment variables? And tell whether these fields are mandatory to run the project in pcf.
Does your app require any services to run? Services would be things like a database or message queue. If it does not, then you do not need to specify any services in your manifest. They are optional.
Similarly, for environment variables, you would only need to set them if they are required to configure your application. Otherwise, just omit that section of your manifest.
At the end of the day, you should talk with whomever developed the application or read the documentation they produce as that's the only way to know what services or environment variables are required.
In regards to your additional questions:
1)And also I have one more query...like in our application we used python ok! In that we use lots of pacakages say pandas,numpy,scipy and so on...how can I import all the libraries into the PCF ??? Buildpacks will contain version only right?
Correct. The buildpack only includes Python itself. Your dependencies either need to be installed or vendored. To do this for Python, you need to include a requirements.txt file. The buildpack will see this and use pip to install your dependencies.
See the docs for the Python buildpack which explains this in more detail: https://docs.cloudfoundry.org/buildpacks/python/index.html#pushing_apps
2)And also tell me what will be the path for my app name if Java I can enclose jar files
For Java apps, you need to push compiled code. That means, you need to run something like mvn package or gradle assemble to build your executable JAR or WAR file. This should be a self contained file that has everything necessary to run your app, compile class files, config, and all dependent JARs.
You then run cf push -p path/to/my-app.jar (or WAR, whatever you build). The cf cli will take everything in the app and push it up to Cloud Foundry where the Java buildpack will install things like the JVM and possibly Tomcat so you app can run.
what should I do for application devloped using pyhton , JavaScript and php....
You can use multiple buildpacks. See the instructions here.
https://docs.cloudfoundry.org/buildpacks/use-multiple-buildpacks.html
In short, you can have as many buildpacks as you want. The last buildpack in the list is special because that is the buildpack which will set the start command for your application (although you can override this with cf push -c if necessary). The non-final buildpacks will run and simply install dependencies.
3) we were using postgresql how can I use this in pcf with my app
Run cf marketplace and see if there are any Postgres providers in your Marketplace. If there is one, you can just do a cf create-service <provider> <plan> <service name> and the foundation will create a database for you to use. You would then run a cf bind-service <app> <service name> to bind the service you create to your app. This will generate credentials and pass them along to your app when it starts. You app can then read the credentials out of VCAP_SERVICES and use them to make connections to the database.
See here for more details:
https://docs.cloudfoundry.org/devguide/services/application-binding.html
https://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES
What kind of tasks or custom commands could be used to generate artifacts for rptproj projects during TFS build process and then to publish the reports to local ReportServer later during deployment process?
The long story.
I have a setup with separate Build and Deployment pipelines in TFS 2018 Express.
My Visual Studio 2017 solution has three ASP.NET MVC web app projects and two SSRS projects (rptproj).
Currently, I'm triggering a build using the following MSBuild parameters:
/p:DeployOnBuild=true /p:WebPublishMethod=Package
/p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true
/p:PackageLocation="$(build.artifactstagingdirectory)\\"
/p:AutoParameterizationWebConfigConnectionStrings=false
and also I have specified an output folder for artifacts.
After a build, the artifact folder contains three zip files for the websites and some deployment helper scripts. However, there is no any output from my rptproj projects (and I'm not sure what should the output be for these). The report projects are built without any issues, judging from the build log, they just don't generate any artifact outputs.
I have configured the deploy pipeline to take the zips and deploy to local IIS server. It works just fine, also web.config transforms get applied.
I did it all using TFS 2018 built-in tasks and everything seemed mostly straight forward. But there are no built-in tasks for SSRS.
I found some third party SSRS tasks in the TFS online gallery but, if I understand correctly, they all seem to be deploying immediately during build. I don't need that - I want first to collect the rptproj output in the artifacts and then later deploy it during the deploy pipeline step because sometimes I want to trigger a manual release for some older build without actually rebuilding everything.
I want first to collect the rptproj output in the artifacts and then later deploy it during the deploy pipeline step
You just stated the exact solution to the problem:
Add a Publish Artifacts task, point it to the folder containing your report output, and publish your reports as an artifact. Then it will be available in a release. In my experience, you don't need to "build" report projects, you can just publish the folder containing your reports from $(Build.SourcesDirectory)/Path/To/Your/Reports.
I have a Windows Store Application (Windows 8.1), hosted on a GIT repo on Visual Studio Online.
I created a build definition, left all the default values as they were, ran the build and downloaded the artifacts.
I didn't find the Powershell script used to side load (install) the application, instead I found an .exe file.
What can be missing to generate the Powershell script needed to install the app?
The default build configuration is copying the files in "bin" folder to artifacts, that's why you see exe file.
To copy the package files to artifacts, please configure your build definition as following:
In "Visual Studio Build" step, add following argument in "MSBuild Arguments":
/p:AppxPackageDir="$(Build.BinariesDirectory)\AppxPackages\\"
And in "Publish Build Artifacts" steps, set "Path to Publish" as following:
$(Build.BinariesDirectory)\AppxPackages
To package Windows Store App during the TFS build process, you can (assume you're using XAML build):
1). Set MSBuild Arguments to be: /p:DeployOnBuild=true;DeployMethod=Package /p:DefaultPackageOutputDir="$(TF_BUILD_BINARIESDIRECTORY)"\StoreAppPackage
2). Set Output location to be 'SingleFolder' or 'PerProject'.
Then, after you queue one build, you will find one folder called StoreAppPackage in the TFS Build Drop folder. You can then find the Add-AppDevPackage.ps1 file.
I need to configure Hudson using SVN and ANT . I need to deploy the war file at my server. Please navigate to the proper help. As I could not find any proper help for this. Do I need to use any plug-in? Or I can do with ANT script? Please suggest a way.
Create a new free-style software project job.
Configure that job.
Under Source Code Management section, select Subversion and configure your repositories and local workspace
Under Build section, select Add build step and select Invoke Ant
If your SVN checkout has build.xml in the root with default target, you don't need anything else. Else configure the Ant targets and the location of the build file
(Optional) Archive the artifacts of the build with Archive the artifacts under Post-build Actions, in your case probably **/*.war
Use Deploy Plugin to publish the war file to remote server.